2024-09-11T04:59:22.433 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T04:59:22.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T04:59:22.538 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194 branch: wip-hemanth-testing-2024-09-10-0723 description: rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-stupid} rados supported-random-distro$/{ubuntu_latest}} email: hyelloji@redhat.com first_in_suite: false job_id: '7900194' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: conf: global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: false bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) mon_bind_msgr2: false sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 9167 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXXq9VNpfhyVQQyDJsSGNqsequkYu3Fn1/5iDqN90uuUBtvYi8ePRQYpfwgyo1CurUoMrPY6c7T6sLULXMbhxY= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:16:14 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T04:59:22.539 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-11T04:59:22.539 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-11T04:59:22.539 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T04:59:22.541 INFO:teuthology.task.internal:Checking packages... 2024-09-11T04:59:22.561 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-11T04:59:22.561 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T04:59:22.561 INFO:teuthology.packaging:ref: None 2024-09-11T04:59:22.561 INFO:teuthology.packaging:tag: None 2024-09-11T04:59:22.561 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-11T04:59:22.561 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T04:59:22.561 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-11T04:59:22.701 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854-g8f219c47-1jammy 2024-09-11T04:59:22.703 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T04:59:22.710 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T04:59:22.710 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T04:59:22.776 INFO:teuthology.task.internal:Saving configuration 2024-09-11T04:59:22.787 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T04:59:22.822 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T04:59:22.847 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 04:53:55.734417', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBOp5+rQKe4Nvp58HaIjkKzw50W5H3U3y3yEBodR1KG/'} 2024-09-11T04:59:22.847 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T04:59:22.864 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-11T04:59:22.864 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T04:59:22.919 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6abf713be0>, signals=[15]) 2024-09-11T04:59:22.919 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T04:59:23.007 INFO:teuthology.task.internal:Opening connections... 2024-09-11T04:59:23.008 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T04:59:23.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T04:59:23.106 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T04:59:23.112 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-09-11T04:59:23.205 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-09-11T04:59:23.206 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-09-11T04:59:23.256 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-11T04:59:23.256 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu" 2024-09-11T04:59:23.256 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="22.04" 2024-09-11T04:59:23.256 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=jammy 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-11T04:59:23.257 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=jammy 2024-09-11T04:59:23.258 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-09-11T04:59:23.272 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T04:59:23.281 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T04:59:23.345 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T04:59:23.346 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-09-11T04:59:23.351 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T04:59:23.360 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T04:59:23.361 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-09-11T04:59:23.410 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T04:59:23.440 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T04:59:23.475 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T04:59:23.475 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T04:59:23.475 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T04:59:23.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T04:59:23.482 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-11T04:59:23.529 INFO:teuthology.orchestra.run.smithi133.stdout:5.15.0-56-generic 2024-09-11T04:59:23.529 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.15.0-56-generic 2024-09-11T04:59:23.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2024-09-11T04:59:23.674 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-09-11T04:59:23.819 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-11T04:59:23.927 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T04:59:24.009 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-11T04:59:24.022 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-11T04:59:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-11T04:59:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-11T04:59:24.129 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-11T04:59:24.129 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-11T04:59:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-11T04:59:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-11T04:59:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-11T04:59:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-11T04:59:24.158 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-11T04:59:24.162 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-11T04:59:24.163 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-11T04:59:24.164 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-11T04:59:24.164 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-11T04:59:24.199 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-11T04:59:24.199 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-11T04:59:24.364 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-11T04:59:24.502 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-11T04:59:24.696 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-11T04:59:24.853 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-11T04:59:24.867 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-11T04:59:24.868 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-11T04:59:24.869 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-11T04:59:24.938 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-11T04:59:24.949 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-11T04:59:24.949 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-11T04:59:24.976 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-11T04:59:25.004 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-11T04:59:25.010 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-11T04:59:25.018 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-11T04:59:25.019 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-11T04:59:25.019 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-11T04:59:25.093 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-11T04:59:25.093 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-11T04:59:25.095 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-11T04:59:25.096 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-11T04:59:25.096 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-11T04:59:25.098 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-11T04:59:25.099 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-11T04:59:25.181 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-11T04:59:25.181 INFO:teuthology.orchestra.run.smithi133.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-11T04:59:30.461 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 15.8 MB in 2s (7,481 kB/s) 2024-09-11T04:59:32.000 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-11T04:59:32.018 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T04:59:32.126 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-11T04:59:32.334 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-09-11T04:59:32.335 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-09-11T04:59:32.520 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T04:59:32.520 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T04:59:32.520 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T04:59:32.521 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T04:59:32.521 INFO:teuthology.orchestra.run.smithi133.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-11T04:59:32.521 INFO:teuthology.orchestra.run.smithi133.stdout: python2.7-minimal 2024-09-11T04:59:32.521 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T04:59:32.522 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2024-09-11T04:59:32.522 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T04:59:32.522 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-11T04:59:32.523 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-11T04:59:32.524 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2024-09-11T04:59:32.524 INFO:teuthology.orchestra.run.smithi133.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-11T04:59:32.524 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2024-09-11T04:59:32.524 INFO:teuthology.orchestra.run.smithi133.stdout: thermald 2024-09-11T04:59:32.583 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2024-09-11T04:59:32.583 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T04:59:32.583 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-11T04:59:32.584 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-11T04:59:32.585 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages will be upgraded: 2024-09-11T04:59:32.585 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-11T04:59:32.765 INFO:teuthology.orchestra.run.smithi133.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T04:59:32.765 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 113 MB of archives. 2024-09-11T04:59:32.766 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-11T04:59:32.766 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-11T04:59:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-11T04:59:34.497 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-11T04:59:36.815 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-11T04:59:36.815 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-11T04:59:36.815 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-11T04:59:37.183 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-11T04:59:37.270 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-11T04:59:37.669 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 113 MB in 5s (24.2 MB/s) 2024-09-11T04:59:37.871 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-11T04:59:39.869 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-11T04:59:39.874 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T04:59:40.118 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T04:59:42.668 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-11T04:59:42.690 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T04:59:42.777 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T04:59:43.321 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-11T04:59:43.342 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T04:59:43.396 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T04:59:49.937 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T04:59:50.045 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T04:59:50.404 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T04:59:50.532 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T04:59:50.852 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-11T04:59:50.875 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-11T04:59:50.918 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T04:59:57.323 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-11T04:59:57.361 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T04:59:57.398 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T05:00:01.211 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T05:00:01.359 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T05:00:01.750 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T05:00:01.914 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T05:00:02.040 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-11T05:00:02.193 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T05:00:03.508 INFO:teuthology.orchestra.run.smithi133.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-11T05:00:03.508 INFO:teuthology.orchestra.run.smithi133.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-11T05:00:03.508 INFO:teuthology.orchestra.run.smithi133.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-11T05:00:03.509 INFO:teuthology.orchestra.run.smithi133.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-11T05:00:03.670 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T05:00:05.113 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T05:00:06.656 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-11T05:00:06.782 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-11T05:00:06.925 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T05:00:06.990 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-11T05:00:06.991 INFO:teuthology.orchestra.run.smithi133.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-11T05:00:21.173 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-11T05:00:21.173 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub' 2024-09-11T05:00:21.200 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T05:00:21.227 INFO:teuthology.orchestra.run.smithi133.stdout:Generating grub configuration file ... 2024-09-11T05:00:21.619 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T05:00:21.647 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T05:00:22.082 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T05:00:22.089 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T05:00:22.157 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T05:00:22.163 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T05:00:22.320 INFO:teuthology.orchestra.run.smithi133.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T05:00:22.320 INFO:teuthology.orchestra.run.smithi133.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T05:00:22.320 INFO:teuthology.orchestra.run.smithi133.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T05:00:22.356 INFO:teuthology.orchestra.run.smithi133.stdout:done 2024-09-11T05:00:22.900 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 21 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2024-09-11T05:00:22.925 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2024-09-11T05:00:22.926 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.15.0.119.119 2024-09-11T05:00:22.926 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T05:00:22.926 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T05:00:22.926 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2024-09-11T05:00:22.926 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2024-09-11T05:00:22.926 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2024-09-11T05:00:22.926 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2024-09-11T05:00:22.926 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-11T05:00:22.926 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T05:00:22.927 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T05:00:22.927 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T05:00:22.927 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T05:00:22.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2024-09-11T05:00:23.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-09-11T05:00:23.392 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T05:00:23.392 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T05:00:23.464 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T05:00:23.545 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T05:00:24.846 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-11T05:00:24.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T05:00:24.927 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-11T05:00:25.124 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-09-11T05:00:25.125 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-09-11T05:00:25.290 INFO:teuthology.orchestra.run.smithi133.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-11T05:00:25.290 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T05:00:25.291 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T05:00:25.291 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T05:00:25.291 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T05:00:25.291 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T05:00:25.291 INFO:teuthology.orchestra.run.smithi133.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T05:00:25.292 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T05:00:25.330 INFO:teuthology.orchestra.run.smithi133.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T05:00:25.332 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2024-09-11T05:00:25.343 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 21 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.15.0.119.119 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2024-09-11T05:00:25.344 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2024-09-11T05:00:25.345 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2024-09-11T05:00:25.345 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2024-09-11T05:00:25.345 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-11T05:00:25.391 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.Gvx10kuyYL 2024-09-11T05:00:25.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Gvx10kuyYL 2024-09-11T05:00:25.443 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.Gvx10kuyYL 2024-09-11T05:00:25.560 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.Gvx10kuyYL is 10KB 2024-09-11T05:00:25.572 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.Gvx10kuyYL 2024-09-11T05:00:25.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-11T05:00:25.631 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:00:25.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-11T05:00:25.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-11T05:00:25.711 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-11T05:00:25.711 DEBUG:teuthology.orchestra.run.smithi133:> sudo update-grub 2024-09-11T05:00:27.243 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub' 2024-09-11T05:00:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T05:00:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-09-11T05:00:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T05:00:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T05:00:27.687 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T05:00:27.693 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T05:00:27.734 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T05:00:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T05:00:27.877 INFO:teuthology.orchestra.run.smithi133.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T05:00:27.877 INFO:teuthology.orchestra.run.smithi133.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T05:00:27.877 INFO:teuthology.orchestra.run.smithi133.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T05:00:27.887 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-09-11T05:00:27.897 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-09-11T05:00:57.969 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T05:00:57.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T05:00:57.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T05:01:58.025 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T05:02:07.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T05:02:07.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T05:02:07.039 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-11T05:02:19.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T05:02:19.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T05:02:19.417 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-11T05:02:20.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T05:02:20.025 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-119-generic"... 2024-09-11T05:02:20.030 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-11T05:02:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:5.15.0-119-generic 2024-09-11T05:02:20.072 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-11T05:02:20.073 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T05:02:20.073 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-11T05:02:21.074 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T05:02:21.074 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T05:02:21.151 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2024-09-11T05:02:21.170 DEBUG:teuthology.parallel:result is None 2024-09-11T05:02:21.170 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T05:02:21.178 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T05:02:21.179 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T05:02:21.184 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T05:02:21.202 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T05:02:21.211 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T05:02:21.211 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T05:02:21.246 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T05:02:21.253 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T05:02:21.254 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T05:02:21.305 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T05:02:21.312 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T05:02:21.314 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T05:02:21.319 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T05:02:21.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T05:02:21.369 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T05:02:21.383 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T05:02:21.383 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T05:02:21.417 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T05:02:21.467 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T05:02:21.516 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:02:21.517 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T05:02:21.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-09-11T05:02:21.658 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T05:02:21.668 INFO:teuthology.task.internal:Starting timer... 2024-09-11T05:02:21.669 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T05:02:21.680 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T05:02:21.690 DEBUG:teuthology.task.selinux:Excluding smithi133: OS 'ubuntu' does not support SELinux 2024-09-11T05:02:21.691 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T05:02:21.691 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-11T05:02:21.691 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T05:02:21.691 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T05:02:21.702 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T05:02:21.705 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T05:02:21.827 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T05:02:21.841 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T05:02:21.842 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T05:09:44.163 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2024-09-11T05:09:44.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T05:09:44.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T05:09:44.248 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-11T05:09:44.325 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-11T05:09:44.326 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T05:09:44.338 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T05:09:44.338 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T05:09:44.338 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T05:09:44.392 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-11T05:09:44.393 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Command line: ntpd -gq 2024-09-11T05:09:44.393 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: ---------------------------------------------------- 2024-09-11T05:09:44.393 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: ntp-4 is maintained by Network Time Foundation, 2024-09-11T05:09:44.393 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-11T05:09:44.393 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: corporation. Support and training for ntp-4 are 2024-09-11T05:09:44.394 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: available at https://www.nwtime.org/support 2024-09-11T05:09:44.394 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: ---------------------------------------------------- 2024-09-11T05:09:44.394 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: proto: precision = 0.046 usec (-24) 2024-09-11T05:09:44.394 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: basedate set to 2022-02-04 2024-09-11T05:09:44.394 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: gps base set to 2022-02-06 (week 2196) 2024-09-11T05:09:44.394 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T05:09:44.395 INFO:teuthology.orchestra.run.smithi133.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T05:09:44.395 INFO:teuthology.orchestra.run.smithi133.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-11T05:09:44.396 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: restrict ::: KOD does nothing without LIMITED. 2024-09-11T05:09:44.396 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Listen and drop on 0 v6wildcard [::]:123 2024-09-11T05:09:44.396 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-11T05:09:44.396 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Listen normally on 2 lo 127.0.0.1:123 2024-09-11T05:09:44.396 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Listen normally on 3 enp3s0f1 172.21.15.133:123 2024-09-11T05:09:44.396 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Listen normally on 4 lo [::1]:123 2024-09-11T05:09:44.397 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8087%5]:123 2024-09-11T05:09:44.397 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:44 ntpd[18458]: Listening on routing socket on fd #22 for interface updates 2024-09-11T05:09:45.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:45.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:45 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:46.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:46 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:46.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:46 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:47 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:48.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:48 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:48.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:48 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:48.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:48 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:48.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:48 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:48 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:48 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:49.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:49.394 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:49 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:50 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:50 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:50 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:50 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:50 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:50.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:50 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:51.393 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T05:09:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T05:09:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T05:09:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T05:09:51.394 INFO:teuthology.orchestra.run.smithi133.stdout:11 Sep 05:09:51 ntpd[18458]: ntpd: time slew +0.000347 s 2024-09-11T05:09:51.394 INFO:teuthology.orchestra.run.smithi133.stdout:ntpd: time slew +0.000347s 2024-09-11T05:09:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-11T05:09:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:11 Sep 05:09:51 ntpd[18458]: can't open /var/log/ntpstats/loopstats.20240911: Permission denied 2024-09-11T05:09:51.451 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T05:09:51.451 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2024-09-11T05:09:51.451 INFO:teuthology.orchestra.run.smithi133.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T05:09:51.451 INFO:teuthology.orchestra.run.smithi133.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T05:09:51.451 INFO:teuthology.orchestra.run.smithi133.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T05:09:51.451 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T05:09:51.452 INFO:teuthology.run_tasks:Running task install... 2024-09-11T05:09:51.462 DEBUG:teuthology.task.install:project ceph 2024-09-11T05:09:51.462 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-11T05:09:51.462 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-11T05:09:51.462 INFO:teuthology.task.install:Using flavor: default 2024-09-11T05:09:51.467 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T05:09:51.467 INFO:teuthology.task.install:extra packages: [] 2024-09-11T05:09:51.468 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-key list | grep Ceph 2024-09-11T05:09:51.552 INFO:teuthology.orchestra.run.smithi133.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-11T05:09:51.581 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-11T05:09:51.582 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph.com (release key) 2024-09-11T05:09:51.582 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-11T05:09:51.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T05:09:51.739 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-11T05:09:51.739 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-11T05:09:51.853 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:09:51.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-11T05:09:51.868 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-09-11T05:09:52.044 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T05:09:52.052 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T05:09:52.069 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T05:09:52.161 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T05:09:52.211 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-11T05:09:52.261 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-11T05:09:52.313 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-11T05:09:52.363 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-11T05:09:52.418 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-11T05:09:52.508 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 29.3 kB in 0s (65.9 kB/s) 2024-09-11T05:09:53.498 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-11T05:09:53.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy 2024-09-11T05:09:53.577 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-11T05:09:53.785 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-09-11T05:09:53.786 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-09-11T05:09:53.978 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T05:09:53.978 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T05:09:53.978 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T05:09:53.978 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T05:09:53.978 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T05:09:53.979 INFO:teuthology.orchestra.run.smithi133.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T05:09:53.979 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T05:09:53.980 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2024-09-11T05:09:53.980 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-11T05:09:53.980 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-11T05:09:53.981 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet zip 2024-09-11T05:09:53.982 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2024-09-11T05:09:53.983 INFO:teuthology.orchestra.run.smithi133.stdout: btrfs-tools 2024-09-11T05:09:54.089 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2024-09-11T05:09:54.089 INFO:teuthology.orchestra.run.smithi133.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-11T05:09:54.089 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-11T05:09:54.089 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-11T05:09:54.089 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-11T05:09:54.089 INFO:teuthology.orchestra.run.smithi133.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-11T05:09:54.089 INFO:teuthology.orchestra.run.smithi133.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-11T05:09:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-11T05:09:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-11T05:09:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-11T05:09:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-11T05:09:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-11T05:09:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T05:09:54.091 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-11T05:09:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-11T05:09:54.151 INFO:teuthology.orchestra.run.smithi133.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-11T05:09:54.151 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 251 MB of archives. 2024-09-11T05:09:54.151 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-11T05:09:54.151 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-11T05:09:54.240 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-11T05:09:54.243 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-11T05:09:54.246 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-11T05:09:54.260 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-11T05:09:54.319 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-11T05:09:54.324 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-11T05:09:54.336 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-11T05:09:54.340 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-11T05:09:54.341 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-11T05:09:54.342 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-11T05:09:54.343 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-11T05:09:54.351 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-11T05:09:54.353 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-11T05:09:54.355 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-11T05:09:54.358 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-11T05:09:54.358 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-11T05:09:54.375 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-11T05:09:54.545 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-11T05:09:54.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-11T05:09:54.547 INFO:teuthology.orchestra.run.smithi133.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-11T05:09:54.548 INFO:teuthology.orchestra.run.smithi133.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-11T05:09:54.548 INFO:teuthology.orchestra.run.smithi133.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-11T05:09:54.548 INFO:teuthology.orchestra.run.smithi133.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-11T05:09:54.548 INFO:teuthology.orchestra.run.smithi133.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-11T05:09:54.548 INFO:teuthology.orchestra.run.smithi133.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-11T05:09:54.550 INFO:teuthology.orchestra.run.smithi133.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-11T05:09:54.550 INFO:teuthology.orchestra.run.smithi133.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-11T05:09:54.550 INFO:teuthology.orchestra.run.smithi133.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-11T05:09:54.565 INFO:teuthology.orchestra.run.smithi133.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-11T05:09:54.565 INFO:teuthology.orchestra.run.smithi133.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-11T05:09:54.565 INFO:teuthology.orchestra.run.smithi133.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-11T05:09:54.566 INFO:teuthology.orchestra.run.smithi133.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-11T05:09:54.566 INFO:teuthology.orchestra.run.smithi133.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-11T05:09:54.567 INFO:teuthology.orchestra.run.smithi133.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-11T05:09:54.581 INFO:teuthology.orchestra.run.smithi133.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-11T05:09:54.582 INFO:teuthology.orchestra.run.smithi133.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-11T05:09:54.585 INFO:teuthology.orchestra.run.smithi133.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-11T05:09:54.599 INFO:teuthology.orchestra.run.smithi133.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-11T05:09:54.601 INFO:teuthology.orchestra.run.smithi133.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-11T05:09:54.601 INFO:teuthology.orchestra.run.smithi133.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-11T05:09:54.604 INFO:teuthology.orchestra.run.smithi133.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-11T05:09:54.606 INFO:teuthology.orchestra.run.smithi133.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-11T05:09:54.607 INFO:teuthology.orchestra.run.smithi133.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-11T05:09:54.608 INFO:teuthology.orchestra.run.smithi133.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-11T05:09:54.616 INFO:teuthology.orchestra.run.smithi133.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-11T05:09:54.617 INFO:teuthology.orchestra.run.smithi133.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-11T05:09:54.620 INFO:teuthology.orchestra.run.smithi133.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-11T05:09:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-11T05:09:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-11T05:09:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-11T05:09:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-11T05:09:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-11T05:09:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-11T05:09:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-11T05:09:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-11T05:09:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-11T05:09:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-11T05:09:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-11T05:09:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-11T05:09:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-11T05:09:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-11T05:09:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-11T05:09:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-11T05:09:54.860 INFO:teuthology.orchestra.run.smithi133.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-11T05:09:54.862 INFO:teuthology.orchestra.run.smithi133.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-11T05:09:54.873 INFO:teuthology.orchestra.run.smithi133.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-11T05:09:55.638 INFO:teuthology.orchestra.run.smithi133.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-11T05:09:55.727 INFO:teuthology.orchestra.run.smithi133.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-11T05:09:55.731 INFO:teuthology.orchestra.run.smithi133.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-11T05:09:55.733 INFO:teuthology.orchestra.run.smithi133.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-11T05:09:55.767 INFO:teuthology.orchestra.run.smithi133.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-11T05:09:55.896 INFO:teuthology.orchestra.run.smithi133.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-11T05:09:56.426 INFO:teuthology.orchestra.run.smithi133.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-11T05:09:56.427 INFO:teuthology.orchestra.run.smithi133.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-11T05:09:56.439 INFO:teuthology.orchestra.run.smithi133.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-11T05:09:56.472 INFO:teuthology.orchestra.run.smithi133.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-11T05:09:56.481 INFO:teuthology.orchestra.run.smithi133.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-11T05:09:56.483 INFO:teuthology.orchestra.run.smithi133.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-11T05:09:57.830 INFO:teuthology.orchestra.run.smithi133.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-11T05:09:57.939 INFO:teuthology.orchestra.run.smithi133.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-11T05:09:57.939 INFO:teuthology.orchestra.run.smithi133.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-11T05:09:59.173 INFO:teuthology.orchestra.run.smithi133.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-11T05:09:59.175 INFO:teuthology.orchestra.run.smithi133.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-11T05:09:59.177 INFO:teuthology.orchestra.run.smithi133.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-11T05:09:59.485 INFO:teuthology.orchestra.run.smithi133.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-11T05:10:00.081 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 251 MB in 5s (46.6 MB/s) 2024-09-11T05:10:00.254 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-11T05:10:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-11T05:10:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-11T05:10:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T05:10:00.781 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-11T05:10:00.807 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-11T05:10:00.848 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T05:10:01.117 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-11T05:10:01.143 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-11T05:10:01.175 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T05:10:01.637 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-11T05:10:01.664 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T05:10:01.712 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T05:10:02.115 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-11T05:10:02.141 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T05:10:02.182 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T05:10:02.459 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-11T05:10:02.485 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T05:10:02.517 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T05:10:02.894 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-11T05:10:02.920 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-11T05:10:03.012 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T05:10:03.323 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librados2. 2024-09-11T05:10:03.349 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../007-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:03.390 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:03.818 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libnbd0. 2024-09-11T05:10:03.843 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-11T05:10:03.884 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-11T05:10:04.137 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librbd1. 2024-09-11T05:10:04.163 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../009-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:04.195 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:04.631 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs2. 2024-09-11T05:10:04.657 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:04.698 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:04.983 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rados. 2024-09-11T05:10:05.010 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:05.050 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:05.344 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-11T05:10:05.371 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:05.402 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:05.605 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cephfs. 2024-09-11T05:10:05.620 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:05.655 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:05.966 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-11T05:10:05.992 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:06.032 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:06.352 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-11T05:10:06.378 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-11T05:10:06.418 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T05:10:06.696 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-prettytable. 2024-09-11T05:10:06.722 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-11T05:10:06.763 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-11T05:10:07.015 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rbd. 2024-09-11T05:10:07.030 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:07.073 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:07.443 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-11T05:10:07.469 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-11T05:10:07.701 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T05:10:08.004 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-11T05:10:08.019 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-11T05:10:08.054 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T05:10:08.348 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-11T05:10:08.374 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-11T05:10:08.414 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T05:10:08.718 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua5.1. 2024-09-11T05:10:08.744 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-11T05:10:08.784 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-11T05:10:09.104 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua-any. 2024-09-11T05:10:09.119 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-11T05:10:09.162 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-11T05:10:09.398 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package zip. 2024-09-11T05:10:09.413 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-11T05:10:09.456 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking zip (3.0-12build2) ... 2024-09-11T05:10:09.767 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package unzip. 2024-09-11T05:10:09.794 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-11T05:10:09.834 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-11T05:10:10.205 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package luarocks. 2024-09-11T05:10:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-11T05:10:10.271 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-11T05:10:10.662 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librgw2. 2024-09-11T05:10:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../026-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:10.728 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:11.306 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rgw. 2024-09-11T05:10:11.321 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:11.725 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-11T05:10:11.752 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-11T05:10:11.791 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T05:10:12.069 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libradosstriper1. 2024-09-11T05:10:12.096 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:12.136 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:12.556 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-common. 2024-09-11T05:10:12.582 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:12.622 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:14.111 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-base. 2024-09-11T05:10:14.138 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:14.195 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:14.706 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-11T05:10:14.722 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-11T05:10:14.756 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-11T05:10:15.092 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cheroot. 2024-09-11T05:10:15.108 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-11T05:10:15.159 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T05:10:15.470 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-11T05:10:15.497 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-11T05:10:15.537 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-11T05:10:15.840 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-11T05:10:15.866 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-11T05:10:15.906 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-11T05:10:16.175 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-11T05:10:16.202 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-11T05:10:16.233 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-11T05:10:16.536 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-tempora. 2024-09-11T05:10:16.562 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-11T05:10:16.603 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-11T05:10:16.906 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-portend. 2024-09-11T05:10:16.932 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-11T05:10:16.972 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-11T05:10:17.283 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-11T05:10:17.310 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-11T05:10:17.350 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-11T05:10:17.611 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-11T05:10:17.637 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-11T05:10:17.678 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-11T05:10:18.156 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-natsort. 2024-09-11T05:10:18.182 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-11T05:10:18.239 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-11T05:10:18.542 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-logutils. 2024-09-11T05:10:18.569 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-11T05:10:18.600 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-11T05:10:18.878 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-11T05:10:18.904 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-11T05:10:18.945 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-11T05:10:19.247 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-mako. 2024-09-11T05:10:19.274 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-11T05:10:19.314 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T05:10:19.634 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-11T05:10:19.660 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-11T05:10:19.700 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-11T05:10:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-11T05:10:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-11T05:10:20.078 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-11T05:10:20.356 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webob. 2024-09-11T05:10:20.382 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-11T05:10:20.422 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T05:10:20.733 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-waitress. 2024-09-11T05:10:20.760 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-11T05:10:20.795 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T05:10:21.078 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-tempita. 2024-09-11T05:10:21.104 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-11T05:10:21.144 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T05:10:21.447 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-paste. 2024-09-11T05:10:21.474 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-11T05:10:21.514 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T05:10:21.867 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-11T05:10:21.882 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-11T05:10:21.925 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T05:10:22.228 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-11T05:10:22.254 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-11T05:10:22.294 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-11T05:10:22.592 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webtest. 2024-09-11T05:10:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-11T05:10:22.655 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-11T05:10:22.966 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pecan. 2024-09-11T05:10:22.994 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-11T05:10:23.033 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T05:10:23.373 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-certifi. 2024-09-11T05:10:23.400 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-11T05:10:23.439 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-11T05:10:23.776 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-idna. 2024-09-11T05:10:23.803 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-11T05:10:23.842 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T05:10:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-urllib3. 2024-09-11T05:10:24.181 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-11T05:10:24.211 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T05:10:24.498 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-requests. 2024-09-11T05:10:24.525 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-11T05:10:24.556 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T05:10:24.876 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-11T05:10:24.903 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-11T05:10:24.951 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T05:10:25.246 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-11T05:10:25.273 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:25.304 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:25.675 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-11T05:10:25.702 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:26.020 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-11T05:10:26.047 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-11T05:10:26.077 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-11T05:10:26.355 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-openssl. 2024-09-11T05:10:26.382 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-11T05:10:26.430 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-11T05:10:26.775 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr. 2024-09-11T05:10:26.802 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:26.841 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:27.245 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mon. 2024-09-11T05:10:27.271 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:27.311 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:28.915 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-osd. 2024-09-11T05:10:28.942 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:28.981 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:30.001 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph. 2024-09-11T05:10:30.018 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../067-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:30.059 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-fuse. 2024-09-11T05:10:30.405 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:30.445 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:30.823 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mds. 2024-09-11T05:10:30.850 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:30.890 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:31.234 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephadm. 2024-09-11T05:10:31.250 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../070-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:31.284 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:31.603 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-11T05:10:31.630 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-11T05:10:31.670 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-11T05:10:32.015 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-11T05:10:32.041 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T05:10:32.081 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T05:10:33.036 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-babel. 2024-09-11T05:10:33.063 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T05:10:33.102 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T05:10:33.447 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jinja2. 2024-09-11T05:10:33.474 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-11T05:10:33.513 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T05:10:33.908 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-11T05:10:33.935 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:33.974 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:34.311 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-11T05:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-11T05:10:34.378 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-11T05:10:34.680 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-routes. 2024-09-11T05:10:34.708 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-11T05:10:34.755 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T05:10:35.084 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-11T05:10:35.111 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:35.167 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:37.746 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-11T05:10:37.763 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-11T05:10:37.804 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T05:10:38.300 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-joblib. 2024-09-11T05:10:38.327 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-11T05:10:38.366 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T05:10:38.866 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-11T05:10:38.893 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-11T05:10:38.944 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-11T05:10:39.224 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn. 2024-09-11T05:10:39.249 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-11T05:10:39.382 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T05:10:40.075 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-11T05:10:40.103 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:40.141 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:41.069 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cachetools. 2024-09-11T05:10:41.097 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-11T05:10:41.136 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-11T05:10:41.438 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-11T05:10:41.466 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-11T05:10:41.505 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-11T05:10:41.875 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-11T05:10:41.903 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-11T05:10:41.942 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T05:10:42.253 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rsa. 2024-09-11T05:10:42.281 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-11T05:10:42.319 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-11T05:10:42.607 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-google-auth. 2024-09-11T05:10:42.634 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-11T05:10:42.665 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-11T05:10:42.951 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-11T05:10:42.979 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-11T05:10:43.018 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T05:10:43.346 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-websocket. 2024-09-11T05:10:43.373 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-11T05:10:43.412 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-11T05:10:43.741 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-11T05:10:43.769 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-11T05:10:43.807 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T05:10:44.454 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-11T05:10:44.470 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:44.512 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:44.840 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-11T05:10:44.868 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-11T05:10:44.907 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T05:10:45.235 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-11T05:10:45.262 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T05:10:45.301 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T05:10:45.562 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package jq. 2024-09-11T05:10:45.590 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T05:10:45.629 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-11T05:10:45.898 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package socat. 2024-09-11T05:10:45.914 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-11T05:10:45.956 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-11T05:10:46.251 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package xmlstarlet. 2024-09-11T05:10:46.267 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-11T05:10:46.309 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-11T05:10:46.672 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-test. 2024-09-11T05:10:46.700 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:46.738 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:49.559 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-volume. 2024-09-11T05:10:49.588 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T05:10:49.626 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:49.930 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-11T05:10:49.957 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:49.996 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:50.307 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-11T05:10:50.335 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-11T05:10:50.373 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T05:10:50.703 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-11T05:10:50.762 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-11T05:10:50.769 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-11T05:10:51.140 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package nvme-cli. 2024-09-11T05:10:51.168 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-11T05:10:51.207 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T05:10:51.620 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package pkg-config. 2024-09-11T05:10:51.648 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-11T05:10:51.686 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T05:10:51.981 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-11T05:10:52.009 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-11T05:10:52.048 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-11T05:10:52.534 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-11T05:10:52.562 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-11T05:10:52.600 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-11T05:10:52.903 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastescript. 2024-09-11T05:10:52.931 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-11T05:10:52.969 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-11T05:10:53.298 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pluggy. 2024-09-11T05:10:53.326 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-11T05:10:53.364 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-11T05:10:53.625 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-psutil. 2024-09-11T05:10:53.642 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-11T05:10:53.675 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-11T05:10:53.995 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-py. 2024-09-11T05:10:54.023 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-11T05:10:54.061 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-11T05:10:54.407 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pygments. 2024-09-11T05:10:54.435 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-11T05:10:54.473 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T05:10:54.944 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-11T05:10:54.972 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-11T05:10:55.010 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-11T05:10:55.321 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-toml. 2024-09-11T05:10:55.349 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-11T05:10:55.387 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-11T05:10:55.699 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pytest. 2024-09-11T05:10:55.727 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-11T05:10:55.765 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T05:10:56.086 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplejson. 2024-09-11T05:10:56.114 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-11T05:10:56.152 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-11T05:10:56.413 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-11T05:10:56.429 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-11T05:10:56.471 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-11T05:10:57.167 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package radosgw. 2024-09-11T05:10:57.195 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../117-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:57.233 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:58.128 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package rbd-fuse. 2024-09-11T05:10:58.141 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T05:10:58.186 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:10:58.507 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-11T05:10:58.758 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T05:10:58.883 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T05:10:59.147 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-11T05:10:59.456 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-11T05:10:59.866 INFO:teuthology.orchestra.run.smithi133.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-11T05:10:59.927 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:00.493 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user cephadm....done 2024-09-11T05:11:00.641 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T05:11:00.918 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-11T05:11:01.169 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-11T05:11:01.295 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-11T05:11:01.554 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-11T05:11:01.813 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T05:11:01.940 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-11T05:11:02.241 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T05:11:02.635 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-11T05:11:02.894 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-11T05:11:03.171 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-11T05:11:03.489 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-11T05:11:03.732 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-11T05:11:03.991 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-11T05:11:04.242 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:04.576 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T05:11:04.702 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-11T05:11:04.828 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T05:11:04.954 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T05:11:05.089 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-11T05:11:05.332 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-11T05:11:05.884 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-11T05:11:05.912 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-11T05:11:05.976 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T05:11:06.102 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-11T05:11:06.478 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T05:11:06.604 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-11T05:11:06.874 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T05:11:07.150 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-11T05:11:07.476 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up zip (3.0-12build2) ... 2024-09-11T05:11:07.602 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T05:11:08.271 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T05:11:08.521 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T05:11:08.623 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-11T05:11:08.723 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T05:11:09.025 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-11T05:11:09.401 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-11T05:11:09.652 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T05:11:10.012 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T05:11:10.304 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T05:11:10.648 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-11T05:11:10.882 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-11T05:11:11.000 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:11.284 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T05:11:11.586 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T05:11:12.758 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T05:11:13.001 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T05:11:13.127 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-11T05:11:13.394 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T05:11:13.520 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-11T05:11:13.646 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T05:11:13.948 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-11T05:11:14.207 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-11T05:11:14.458 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T05:11:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-11T05:11:14.877 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-11T05:11:15.136 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-11T05:11:15.396 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-11T05:11:15.656 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-11T05:11:15.906 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-11T05:11:16.165 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T05:11:16.283 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-11T05:11:16.542 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T05:11:16.652 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T05:11:16.969 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T05:11:17.262 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-11T05:11:17.530 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T05:11:17.656 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-11T05:11:17.782 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T05:11:17.871 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-11T05:11:18.100 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T05:11:18.201 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T05:11:18.586 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-11T05:11:18.845 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-11T05:11:18.971 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-11T05:11:19.231 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T05:11:19.356 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T05:11:19.683 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T05:11:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T05:11:20.277 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-11T05:11:20.402 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-11T05:11:20.687 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-11T05:11:21.048 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T05:11:21.316 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-11T05:11:21.743 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-11T05:11:22.027 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T05:11:22.380 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T05:11:22.506 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:22.640 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:22.766 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-11T05:11:23.078 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:23.203 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:23.329 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-11T05:11:23.606 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:23.732 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:23.858 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:24.058 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T05:11:24.058 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T05:11:24.543 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:24.669 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:24.794 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:24.921 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T05:11:26.169 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:26.275 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:26.401 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:26.527 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:26.653 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:27.196 INFO:teuthology.orchestra.run.smithi133.stdout:Adding group ceph....done 2024-09-11T05:11:27.581 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user ceph....done 2024-09-11T05:11:27.741 INFO:teuthology.orchestra.run.smithi133.stdout:Setting system user ceph properties....done 2024-09-11T05:11:27.755 INFO:teuthology.orchestra.run.smithi133.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-11T05:11:27.891 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-11T05:11:28.240 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-11T05:11:28.695 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:28.821 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:29.288 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T05:11:29.288 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T05:11:29.756 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:30.100 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-11T05:11:30.550 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:30.819 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T05:11:30.819 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T05:11:31.296 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:31.512 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T05:11:31.512 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T05:11:32.011 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:32.280 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T05:11:32.281 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T05:11:32.746 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:32.872 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:33.023 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:33.234 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T05:11:33.234 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T05:11:33.691 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:33.817 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:33.944 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T05:11:34.446 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T05:11:34.614 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T05:11:34.766 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T05:11:35.884 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T05:11:36.545 INFO:teuthology.orchestra.run.smithi133.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T05:11:36.551 DEBUG:teuthology.parallel:result is None 2024-09-11T05:11:36.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T05:11:36.695 DEBUG:teuthology.orchestra.run.smithi133:> dpkg-query -W -f '${Version}' ceph 2024-09-11T05:11:36.714 INFO:teuthology.orchestra.run.smithi133.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-11T05:11:36.714 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-11T05:11:36.715 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-11T05:11:36.716 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T05:11:36.717 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:36.717 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T05:11:36.769 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T05:11:36.769 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:36.769 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T05:11:36.824 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T05:11:36.876 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T05:11:36.876 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:36.876 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T05:11:36.929 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T05:11:36.985 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T05:11:36.986 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:36.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T05:11:37.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T05:11:37.097 INFO:teuthology.run_tasks:Running task ceph... 2024-09-11T05:11:37.177 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-11T05:11:37.177 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-09-11T05:11:37.190 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-11T05:11:37.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T05:11:37.244 INFO:tasks.ceph:Creating extra log directories... 2024-09-11T05:11:37.244 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T05:11:37.305 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-11T05:11:37.305 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-11T05:11:37.305 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-stupid} rados supported-random-distro$/{ubuntu_latest}}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7900194', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 9167, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'targets': {'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXXq9VNpfhyVQQyDJsSGNqsequkYu3Fn1/5iDqN90uuUBtvYi8ePRQYpfwgyo1CurUoMrPY6c7T6sLULXMbhxY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'mon_bind_msgr2': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_17:16:14', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-11T05:11:37.306 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T05:11:37.345 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T05:11:37.402 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:37.402 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T05:11:37.450 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T05:11:37.450 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-09-11T05:11:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T05:11:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T05:11:37.498 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-11T05:11:37.498 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T05:11:37.498 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 05:06:17.210728704 +0000 2024-09-11T05:11:37.498 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 05:06:16.954730976 +0000 2024-09-11T05:11:37.498 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 05:06:16.954730976 +0000 2024-09-11T05:11:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-11T05:11:37.499 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T05:11:37.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T05:11:37.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T05:11:37.552 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000385082 s, 1.3 MB/s 2024-09-11T05:11:37.553 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T05:11:37.603 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-09-11T05:11:37.649 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T05:11:37.649 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T05:11:37.649 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-09-11T05:11:37.649 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T05:11:37.649 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 05:06:17.674724584 +0000 2024-09-11T05:11:37.650 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 05:06:17.434726716 +0000 2024-09-11T05:11:37.650 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 05:06:17.434726716 +0000 2024-09-11T05:11:37.650 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-11T05:11:37.650 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T05:11:37.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T05:11:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T05:11:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000447089 s, 1.1 MB/s 2024-09-11T05:11:37.705 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T05:11:37.755 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-09-11T05:11:37.801 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T05:11:37.802 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T05:11:37.802 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-09-11T05:11:37.802 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T05:11:37.802 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 05:06:18.190720004 +0000 2024-09-11T05:11:37.802 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 05:06:17.946722170 +0000 2024-09-11T05:11:37.802 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 05:06:17.946722170 +0000 2024-09-11T05:11:37.802 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-11T05:11:37.802 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T05:11:37.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T05:11:37.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T05:11:37.857 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000412437 s, 1.2 MB/s 2024-09-11T05:11:37.858 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T05:11:37.906 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-09-11T05:11:37.953 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T05:11:37.954 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T05:11:37.954 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-09-11T05:11:37.954 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T05:11:37.954 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-11 05:06:18.666715778 +0000 2024-09-11T05:11:37.954 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-11 05:06:18.418717979 +0000 2024-09-11T05:11:37.954 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-11 05:06:18.418717979 +0000 2024-09-11T05:11:37.954 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-11T05:11:37.954 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T05:11:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-11T05:11:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-11T05:11:38.011 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000389294 s, 1.3 MB/s 2024-09-11T05:11:38.013 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T05:11:38.063 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T05:11:38.063 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-11T05:11:38.063 INFO:tasks.ceph:Generating config... 2024-09-11T05:11:38.064 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-11T05:11:38.064 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-11T05:11:38.064 INFO:tasks.ceph:[global] ms type = async 2024-09-11T05:11:38.064 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-11T05:11:38.064 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-11T05:11:38.064 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-11T05:11:38.065 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-11T05:11:38.065 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-11T05:11:38.065 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] bluestore write v2 = False 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-11T05:11:38.065 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-11T05:11:38.066 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-11T05:11:38.066 INFO:tasks.ceph:Setting up mon.a... 2024-09-11T05:11:38.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-11T05:11:38.136 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.keyring 2024-09-11T05:11:38.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-11T05:11:38.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T05:11:38.285 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.133:6789]'), ('mon.b', '[v1:172.21.15.133:6790]'), ('mon.c', '[v1:172.21.15.133:6791]')] 2024-09-11T05:11:38.286 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.133:6789],[v1:172.21.15.133:6790],[v1:172.21.15.133:6791]', 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-11T05:11:38.287 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:38.287 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T05:11:38.334 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.133:6789]' --addv b '[v1:172.21.15.133:6790]' --addv c '[v1:172.21.15.133:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-11T05:11:38.421 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-11T05:11:38.421 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: generated fsid 40251081-3433-4f55-b0e4-ce0789fd19fd 2024-09-11T05:11:38.421 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = quincy 2024-09-11T05:11:38.422 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 0 2024-09-11T05:11:38.422 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 40251081-3433-4f55-b0e4-ce0789fd19fd 2024-09-11T05:11:38.422 INFO:teuthology.orchestra.run.smithi133.stdout:last_changed 2024-09-11T05:11:38.423932+0000 2024-09-11T05:11:38.422 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-09-11T05:11:38.423932+0000 2024-09-11T05:11:38.422 INFO:teuthology.orchestra.run.smithi133.stdout:min_mon_release 17 (quincy) 2024-09-11T05:11:38.422 INFO:teuthology.orchestra.run.smithi133.stdout:election_strategy: 1 2024-09-11T05:11:38.423 INFO:teuthology.orchestra.run.smithi133.stdout:0: v1:172.21.15.133:6789/0 mon.a 2024-09-11T05:11:38.423 INFO:teuthology.orchestra.run.smithi133.stdout:1: v1:172.21.15.133:6790/0 mon.b 2024-09-11T05:11:38.423 INFO:teuthology.orchestra.run.smithi133.stdout:2: v1:172.21.15.133:6791/0 mon.c 2024-09-11T05:11:38.423 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-11T05:11:38.425 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T05:11:38.473 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 40251081-3433-4f55-b0e4-ce0789fd19fd... 2024-09-11T05:11:38.474 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T05:11:38.548 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.549 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-09-11T05:11:38.550 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-09-11T05:11:38.551 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-09-11T05:11:38.551 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-09-11T05:11:38.551 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-09-11T05:11:38.551 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-09-11T05:11:38.551 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T05:11:38.551 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T05:11:38.551 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "[v1:172.21.15.133:6789],[v1:172.21.15.133:6790],[v1:172.21.15.133:6791]" 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: mon election default strategy = 1 2024-09-11T05:11:38.552 INFO:teuthology.orchestra.run.smithi133.stdout: ms bind msgr2 = False 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: ms type = async 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 40251081-3433-4f55-b0e4-ce0789fd19fd 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.553 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-09-11T05:11:38.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-09-11T05:11:38.555 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-09-11T05:11:38.556 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-09-11T05:11:38.556 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = stupid 2024-09-11T05:11:38.556 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-09-11T05:11:38.556 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-09-11T05:11:38.556 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore write v2 = False 2024-09-11T05:11:38.556 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore zero block detection = True 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-09-11T05:11:38.557 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify cached snaps = True 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify missing on start = True 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock override recovery settings = True 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: osd op queue = debug_random 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: osd op queue cut off = debug_random 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-09-11T05:11:38.558 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-09-11T05:11:38.559 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T05:11:38.560 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: mon scrub interval = 300 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-09-11T05:11:38.561 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-09-11T05:11:38.562 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-09-11T05:11:38.562 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-09-11T05:11:38.562 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T05:11:38.562 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T05:11:38.562 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-09-11T05:11:38.562 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-09-11T05:11:38.562 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-09-11T05:11:38.563 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-11T05:11:38.564 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-11T05:11:38.653 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-11T05:11:38.653 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:38.653 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-11T05:11:38.658 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:38.659 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-11T05:11:38.705 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:11:38.705 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:38.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T05:11:38.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T05:11:38.770 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:38.770 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T05:11:38.822 INFO:tasks.ceph:Setting up mon nodes... 2024-09-11T05:11:38.822 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-11T05:11:38.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T05:11:38.907 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T05:11:38.913 INFO:tasks.ceph:Setting up mds nodes... 2024-09-11T05:11:38.913 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-11T05:11:38.913 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-11T05:11:38.913 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-11T05:11:38.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-11T05:11:38.969 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T05:11:38.969 INFO:tasks.ceph:role: osd.0 2024-09-11T05:11:38.969 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:11:38.969 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T05:11:39.033 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T05:11:39.033 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T05:11:39.033 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T05:11:39.033 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T05:11:39.033 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T05:11:39.034 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-11T05:11:39.034 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T05:11:39.034 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T05:11:39.034 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T05:11:39.034 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T05:11:39.485 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-11T05:11:39.487 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-11T05:11:39.487 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-11T05:11:39.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-11T05:11:39.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:11:39.591 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T05:11:39.592 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-11T05:11:39.642 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T05:11:39.643 INFO:tasks.ceph:role: osd.1 2024-09-11T05:11:39.643 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:11:39.643 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T05:11:39.706 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T05:11:39.706 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T05:11:39.706 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T05:11:39.706 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T05:11:39.706 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T05:11:39.706 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-11T05:11:39.706 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T05:11:39.707 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T05:11:39.707 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T05:11:39.707 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T05:11:40.171 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-11T05:11:40.173 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-11T05:11:40.174 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-11T05:11:40.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-11T05:11:40.306 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T05:11:40.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:11:40.307 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-11T05:11:40.360 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T05:11:40.360 INFO:tasks.ceph:role: osd.2 2024-09-11T05:11:40.360 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:11:40.360 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T05:11:40.423 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T05:11:40.889 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-11T05:11:40.891 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-11T05:11:40.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-11T05:11:41.007 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-11T05:11:41.019 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T05:11:41.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:11:41.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T05:11:41.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:41.136+0000 7f489bb957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-11T05:11:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:41.140+0000 7f489bb957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-11T05:11:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:41.140+0000 7f489bb957c0 -1 bdev(0x559e89845800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-11T05:11:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:41.140+0000 7f489bb957c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-11T05:11:44.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-11T05:11:44.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T05:11:44.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:44.376+0000 7fd1490567c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-11T05:11:44.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:44.376+0000 7fd1490567c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-11T05:11:44.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:44.376+0000 7fd1490567c0 -1 bdev(0x5633e9ed7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-11T05:11:44.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:44.376+0000 7fd1490567c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-11T05:11:47.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-11T05:11:47.589 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T05:11:47.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:47.668+0000 7fbf8df537c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-11T05:11:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:47.668+0000 7fbf8df537c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-11T05:11:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:47.668+0000 7fbf8df537c0 -1 bdev(0x55b8249dd800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-11T05:11:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:11:47.668+0000 7fbf8df537c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-11T05:11:50.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T05:11:50.910 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-11T05:11:50.911 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:50.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-11T05:11:50.964 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:50.965 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-11T05:11:51.020 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:51.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-11T05:11:51.077 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:11:51.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-11T05:11:51.133 INFO:tasks.ceph:Adding keys to all mons... 2024-09-11T05:11:51.133 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T05:11:51.187 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-09-11T05:11:51.187 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCKJuFmfH1ENhAAm09+P/2GlBrjwwnGkzGMKA== 2024-09-11T05:11:51.187 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-09-11T05:11:51.188 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCNJuFmksp3CBAAH40Pr5ONClfjm3ujqXkjbw== 2024-09-11T05:11:51.188 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-09-11T05:11:51.188 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCQJuFm1XjDFhAALMas8hWKigCgZWKWSTqmZA== 2024-09-11T05:11:51.188 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-09-11T05:11:51.188 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCTJuFmP7UqKBAANeJrzlOU9rgaY3btL5N3Vg== 2024-09-11T05:11:51.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T05:11:51.277 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T05:11:51.320 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T05:11:51.404 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T05:11:51.490 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-11T05:11:51.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-11T05:11:51.502 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T05:11:51.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-11T05:11:51.604 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-11T05:11:51.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T05:11:51.741 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-11T05:11:51.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-11T05:11:51.851 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T05:11:51.940 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-11T05:11:51.951 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T05:11:51.997 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-11T05:11:51.997 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-11T05:11:51.997 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-11T05:11:52.042 INFO:tasks.ceph.mon.a:Started 2024-09-11T05:11:52.042 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-11T05:11:52.042 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-11T05:11:52.044 INFO:tasks.ceph.mon.b:Started 2024-09-11T05:11:52.044 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-11T05:11:52.044 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-11T05:11:52.046 INFO:tasks.ceph.mon.c:Started 2024-09-11T05:11:52.046 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph config set mgr mgr_pool false --force 2024-09-11T05:12:02.356 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-11T05:12:02.356 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-11T05:12:02.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-11T05:12:02.359 INFO:tasks.ceph.mgr.x:Started 2024-09-11T05:12:02.359 DEBUG:tasks.ceph:set 0 configs 2024-09-11T05:12:02.359 INFO:tasks.ceph:Setting crush tunables to default 2024-09-11T05:12:02.360 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd crush tunables default 2024-09-11T05:12:02.547 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-09-11T05:12:02.564 INFO:tasks.ceph:check_enable_crimson: False 2024-09-11T05:12:02.565 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-11T05:12:02.565 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:12:02.565 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-11T05:12:02.576 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:12:02.576 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-11T05:12:02.626 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-11T05:12:02.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-11T05:12:02.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new f7b6888c-0033-4334-b02d-34bc88a2cbb6 0 2024-09-11T05:12:02.920 INFO:teuthology.orchestra.run.smithi133.stdout:0 2024-09-11T05:12:02.936 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 0530ffc9-4ed5-4380-929c-e39eab601f4f 1 2024-09-11T05:12:03.140 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-09-11T05:12:03.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 01d3910a-105e-4fa7-93fd-55d913165415 2 2024-09-11T05:12:03.380 INFO:teuthology.orchestra.run.smithi133.stdout:2 2024-09-11T05:12:03.395 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T05:12:03.395 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T05:12:03.397 INFO:tasks.ceph.osd.0:Started 2024-09-11T05:12:03.397 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T05:12:03.398 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T05:12:03.399 INFO:tasks.ceph.osd.1:Started 2024-09-11T05:12:03.400 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T05:12:03.401 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T05:12:03.403 INFO:tasks.ceph.osd.2:Started 2024-09-11T05:12:03.403 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:03.572 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:03.572 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:03.375435+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T05:12:03.375435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:03.587 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-11T05:12:03.588 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-11T05:12:03.889 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:04.065 INFO:teuthology.misc.health.smithi133.stdout: 2024-09-11T05:12:04.065 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":5,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:03.375435+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T05:12:03.375435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:04.079 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-11T05:12:04.770 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-09-11T05:12:04.768+0000 7f84d75ca7c0 -1 Falling back to public interface 2024-09-11T05:12:04.798 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-09-11T05:12:04.796+0000 7f3e7b5547c0 -1 Falling back to public interface 2024-09-11T05:12:05.063 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:12:05.060+0000 7f2c88a2d7c0 -1 Falling back to public interface 2024-09-11T05:12:05.461 INFO:tasks.ceph.mgr.x.smithi133.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-11T05:12:05.462 INFO:tasks.ceph.mgr.x.smithi133.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-11T05:12:05.462 INFO:tasks.ceph.mgr.x.smithi133.stderr: from numpy import show_config as show_numpy_config 2024-09-11T05:12:08.350 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-11T05:12:08.348+0000 7f5919120640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T05:12:08.351 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-11T05:12:08.348+0000 7f5919120640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T05:12:08.800 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-09-11T05:12:08.796+0000 7f84d75ca7c0 -1 osd.1 0 log_to_monitors true 2024-09-11T05:12:08.882 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-09-11T05:12:08.880+0000 7f3e7b5547c0 -1 osd.2 0 log_to_monitors true 2024-09-11T05:12:09.099 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:12:09.096+0000 7f2c88a2d7c0 -1 osd.0 0 log_to_monitors true 2024-09-11T05:12:10.382 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:10.401 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-09-11T05:12:10.400+0000 7f3e774fd640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T05:12:10.402 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:12:10.400+0000 7f2c849d6640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T05:12:10.410 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:12:10.408+0000 7f2c80810640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T05:12:10.414 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-09-11T05:12:10.412+0000 7f3e73337640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T05:12:10.742 INFO:teuthology.misc.health.smithi133.stdout: 2024-09-11T05:12:10.742 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":7,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:10.388890+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T05:12:03.375435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:10.761 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-11T05:12:11.348 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-09-11T05:12:11.344+0000 7f84d3573640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T05:12:11.353 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-09-11T05:12:11.352+0000 7f84cb3a5640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T05:12:17.063 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:17.366 INFO:teuthology.misc.health.smithi133.stdout: 2024-09-11T05:12:17.366 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":9,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:12.409954+0000","last_up_change":"2024-09-11T05:12:11.399069+0000","last_in_change":"2024-09-11T05:12:03.375435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":2165098313}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":2165098313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":2165098313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":2165098313}]},"public_addr":"172.21.15.133:6808/2165098313","cluster_addr":"172.21.15.133:6809/2165098313","heartbeat_back_addr":"172.21.15.133:6811/2165098313","heartbeat_front_addr":"172.21.15.133:6810/2165098313","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":3149929203}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":3149929203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":3149929203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":3149929203}]},"public_addr":"172.21.15.133:6800/3149929203","cluster_addr":"172.21.15.133:6801/3149929203","heartbeat_back_addr":"172.21.15.133:6803/3149929203","heartbeat_front_addr":"172.21.15.133:6802/3149929203","state":["exists","up"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":841703869}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":841703869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":841703869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":841703869}]},"public_addr":"172.21.15.133:6804/841703869","cluster_addr":"172.21.15.133:6805/841703869","heartbeat_back_addr":"172.21.15.133:6807/841703869","heartbeat_front_addr":"172.21.15.133:6806/841703869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:17.383 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-11T05:12:17.383 INFO:tasks.ceph:Creating RBD pool 2024-09-11T05:12:17.384 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-11T05:12:18.454 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'rbd' created 2024-09-11T05:12:18.477 DEBUG:teuthology.orchestra.run.smithi133:> rbd --cluster ceph pool init rbd 2024-09-11T05:12:21.514 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-11T05:12:21.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-11T05:12:21.515 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-11T05:12:21.826 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:21.844 INFO:teuthology.orchestra.run.smithi133.stdout:[{"version":3,"timestamp":"2024-09-11T05:12:11.351983+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"6571.510940"}]}] 2024-09-11T05:12:21.845 INFO:tasks.ceph_manager:config epoch is 3 2024-09-11T05:12:21.845 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T05:12:21.845 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-11T05:12:21.845 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-11T05:12:22.189 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:22.204 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":4,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6812","nonce":1467469070}]},"active_addr":"172.21.15.133:6812/1467469070","active_change":"2024-09-11T05:12:07.325408+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":2609373758}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":740531886}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":244319715}]}]} 2024-09-11T05:12:22.209 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-11T05:12:22.209 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T05:12:22.209 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:22.513 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:22.514 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":13,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:21.467868+0000","last_up_change":"2024-09-11T05:12:11.399069+0000","last_in_change":"2024-09-11T05:12:03.375435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T05:12:17.695239+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":2165098313}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":2165098313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":2165098313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":2165098313}]},"public_addr":"172.21.15.133:6808/2165098313","cluster_addr":"172.21.15.133:6809/2165098313","heartbeat_back_addr":"172.21.15.133:6811/2165098313","heartbeat_front_addr":"172.21.15.133:6810/2165098313","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":3149929203}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":3149929203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":3149929203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":3149929203}]},"public_addr":"172.21.15.133:6800/3149929203","cluster_addr":"172.21.15.133:6801/3149929203","heartbeat_back_addr":"172.21.15.133:6803/3149929203","heartbeat_front_addr":"172.21.15.133:6802/3149929203","state":["exists","up"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":841703869}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":841703869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":841703869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":841703869}]},"public_addr":"172.21.15.133:6804/841703869","cluster_addr":"172.21.15.133:6805/841703869","heartbeat_back_addr":"172.21.15.133:6807/841703869","heartbeat_front_addr":"172.21.15.133:6806/841703869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:22.531 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T05:12:22.531 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:22.805 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:22.805 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":13,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:21.467868+0000","last_up_change":"2024-09-11T05:12:11.399069+0000","last_in_change":"2024-09-11T05:12:03.375435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T05:12:17.695239+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":2165098313}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":2165098313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":2165098313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":2165098313}]},"public_addr":"172.21.15.133:6808/2165098313","cluster_addr":"172.21.15.133:6809/2165098313","heartbeat_back_addr":"172.21.15.133:6811/2165098313","heartbeat_front_addr":"172.21.15.133:6810/2165098313","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":3149929203}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":3149929203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":3149929203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":3149929203}]},"public_addr":"172.21.15.133:6800/3149929203","cluster_addr":"172.21.15.133:6801/3149929203","heartbeat_back_addr":"172.21.15.133:6803/3149929203","heartbeat_front_addr":"172.21.15.133:6802/3149929203","state":["exists","up"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":841703869}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":841703869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":841703869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":841703869}]},"public_addr":"172.21.15.133:6804/841703869","cluster_addr":"172.21.15.133:6805/841703869","heartbeat_back_addr":"172.21.15.133:6807/841703869","heartbeat_front_addr":"172.21.15.133:6806/841703869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:22.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T05:12:22.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T05:12:22.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T05:12:22.974 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:22.975 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:22.991 INFO:teuthology.orchestra.run.smithi133.stdout:34359738372 2024-09-11T05:12:22.992 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:12:23.000 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:23.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:23.273 INFO:teuthology.orchestra.run.smithi133.stdout:34359738371 2024-09-11T05:12:23.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.0 2024-09-11T05:12:23.334 INFO:teuthology.orchestra.run.smithi133.stdout:34359738370 2024-09-11T05:12:23.344 INFO:teuthology.orchestra.run.smithi133.stdout:34359738371 2024-09-11T05:12:23.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738370 for osd.2 2024-09-11T05:12:23.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.1 2024-09-11T05:12:24.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:24.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:12:24.363 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:24.607 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:24.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.0 2024-09-11T05:12:24.624 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:24.650 INFO:teuthology.orchestra.run.smithi133.stdout:34359738372 2024-09-11T05:12:24.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.2 2024-09-11T05:12:24.669 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:24.691 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:24.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.1 2024-09-11T05:12:24.707 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:24.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T05:12:24.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:25.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:25.027 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:25.043 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-11T05:12:23.339907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81420,"kb_used_data":328,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034100,"statfs":{"total":289910292480,"available":289826918400,"internally_reserved":0,"allocated":335872,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.860977"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488874+0000","last_change":"2024-09-11T05:12:21.489349+0000","last_active":"2024-09-11T05:12:21.488874+0000","last_peered":"2024-09-11T05:12:21.488874+0000","last_clean":"2024-09-11T05:12:21.488874+0000","last_became_active":"2024-09-11T05:12:19.468858+0000","last_became_peered":"2024-09-11T05:12:19.468858+0000","last_unstale":"2024-09-11T05:12:21.488874+0000","last_undegraded":"2024-09-11T05:12:21.488874+0000","last_fullsized":"2024-09-11T05:12:21.488874+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:27.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488073+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:21.488073+0000","last_peered":"2024-09-11T05:12:21.488073+0000","last_clean":"2024-09-11T05:12:21.488073+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:21.488073+0000","last_undegraded":"2024-09-11T05:12:21.488073+0000","last_fullsized":"2024-09-11T05:12:21.488073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977786+0000","last_change":"2024-09-11T05:12:21.977786+0000","last_active":"2024-09-11T05:12:21.977786+0000","last_peered":"2024-09-11T05:12:21.977786+0000","last_clean":"2024-09-11T05:12:21.977786+0000","last_became_active":"2024-09-11T05:12:19.481439+0000","last_became_peered":"2024-09-11T05:12:19.481439+0000","last_unstale":"2024-09-11T05:12:21.977786+0000","last_undegraded":"2024-09-11T05:12:21.977786+0000","last_fullsized":"2024-09-11T05:12:21.977786+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:33.453711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057330499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488575+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:21.488575+0000","last_peered":"2024-09-11T05:12:21.488575+0000","last_clean":"2024-09-11T05:12:21.488575+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:21.488575+0000","last_undegraded":"2024-09-11T05:12:21.488575+0000","last_fullsized":"2024-09-11T05:12:21.488575+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487623+0000","last_change":"2024-09-11T05:12:21.488345+0000","last_active":"2024-09-11T05:12:21.487623+0000","last_peered":"2024-09-11T05:12:21.487623+0000","last_clean":"2024-09-11T05:12:21.487623+0000","last_became_active":"2024-09-11T05:12:19.468806+0000","last_became_peered":"2024-09-11T05:12:19.468806+0000","last_unstale":"2024-09-11T05:12:21.487623+0000","last_undegraded":"2024-09-11T05:12:21.487623+0000","last_fullsized":"2024-09-11T05:12:21.487623+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:47.146442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.491456+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:21.491456+0000","last_peered":"2024-09-11T05:12:21.491456+0000","last_clean":"2024-09-11T05:12:21.491456+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:21.491456+0000","last_undegraded":"2024-09-11T05:12:21.491456+0000","last_fullsized":"2024-09-11T05:12:21.491456+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977823+0000","last_change":"2024-09-11T05:12:21.977823+0000","last_active":"2024-09-11T05:12:21.977823+0000","last_peered":"2024-09-11T05:12:21.977823+0000","last_clean":"2024-09-11T05:12:21.977823+0000","last_became_active":"2024-09-11T05:12:19.477467+0000","last_became_peered":"2024-09-11T05:12:19.477467+0000","last_unstale":"2024-09-11T05:12:21.977823+0000","last_undegraded":"2024-09-11T05:12:21.977823+0000","last_fullsized":"2024-09-11T05:12:21.977823+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058685499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487499+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:21.487499+0000","last_peered":"2024-09-11T05:12:21.487499+0000","last_clean":"2024-09-11T05:12:21.487499+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:21.487499+0000","last_undegraded":"2024-09-11T05:12:21.487499+0000","last_fullsized":"2024-09-11T05:12:21.487499+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:25.044 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:25.325 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:25.341 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-11T05:12:23.339907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81420,"kb_used_data":328,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034100,"statfs":{"total":289910292480,"available":289826918400,"internally_reserved":0,"allocated":335872,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.860977"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488874+0000","last_change":"2024-09-11T05:12:21.489349+0000","last_active":"2024-09-11T05:12:21.488874+0000","last_peered":"2024-09-11T05:12:21.488874+0000","last_clean":"2024-09-11T05:12:21.488874+0000","last_became_active":"2024-09-11T05:12:19.468858+0000","last_became_peered":"2024-09-11T05:12:19.468858+0000","last_unstale":"2024-09-11T05:12:21.488874+0000","last_undegraded":"2024-09-11T05:12:21.488874+0000","last_fullsized":"2024-09-11T05:12:21.488874+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:27.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488073+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:21.488073+0000","last_peered":"2024-09-11T05:12:21.488073+0000","last_clean":"2024-09-11T05:12:21.488073+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:21.488073+0000","last_undegraded":"2024-09-11T05:12:21.488073+0000","last_fullsized":"2024-09-11T05:12:21.488073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977786+0000","last_change":"2024-09-11T05:12:21.977786+0000","last_active":"2024-09-11T05:12:21.977786+0000","last_peered":"2024-09-11T05:12:21.977786+0000","last_clean":"2024-09-11T05:12:21.977786+0000","last_became_active":"2024-09-11T05:12:19.481439+0000","last_became_peered":"2024-09-11T05:12:19.481439+0000","last_unstale":"2024-09-11T05:12:21.977786+0000","last_undegraded":"2024-09-11T05:12:21.977786+0000","last_fullsized":"2024-09-11T05:12:21.977786+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:33.453711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057330499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488575+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:21.488575+0000","last_peered":"2024-09-11T05:12:21.488575+0000","last_clean":"2024-09-11T05:12:21.488575+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:21.488575+0000","last_undegraded":"2024-09-11T05:12:21.488575+0000","last_fullsized":"2024-09-11T05:12:21.488575+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487623+0000","last_change":"2024-09-11T05:12:21.488345+0000","last_active":"2024-09-11T05:12:21.487623+0000","last_peered":"2024-09-11T05:12:21.487623+0000","last_clean":"2024-09-11T05:12:21.487623+0000","last_became_active":"2024-09-11T05:12:19.468806+0000","last_became_peered":"2024-09-11T05:12:19.468806+0000","last_unstale":"2024-09-11T05:12:21.487623+0000","last_undegraded":"2024-09-11T05:12:21.487623+0000","last_fullsized":"2024-09-11T05:12:21.487623+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:47.146442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.491456+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:21.491456+0000","last_peered":"2024-09-11T05:12:21.491456+0000","last_clean":"2024-09-11T05:12:21.491456+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:21.491456+0000","last_undegraded":"2024-09-11T05:12:21.491456+0000","last_fullsized":"2024-09-11T05:12:21.491456+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977823+0000","last_change":"2024-09-11T05:12:21.977823+0000","last_active":"2024-09-11T05:12:21.977823+0000","last_peered":"2024-09-11T05:12:21.977823+0000","last_clean":"2024-09-11T05:12:21.977823+0000","last_became_active":"2024-09-11T05:12:19.477467+0000","last_became_peered":"2024-09-11T05:12:19.477467+0000","last_unstale":"2024-09-11T05:12:21.977823+0000","last_undegraded":"2024-09-11T05:12:21.977823+0000","last_fullsized":"2024-09-11T05:12:21.977823+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058685499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487499+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:21.487499+0000","last_peered":"2024-09-11T05:12:21.487499+0000","last_clean":"2024-09-11T05:12:21.487499+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:21.487499+0000","last_undegraded":"2024-09-11T05:12:21.487499+0000","last_fullsized":"2024-09-11T05:12:21.487499+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:25.342 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T05:12:25.342 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T05:12:25.342 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-11T05:12:25.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-11T05:12:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:25.650 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T05:12:25.670 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-11T05:12:25.670 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-09-11T05:12:25.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:26.003 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:26.003 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":13,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:21.467868+0000","last_up_change":"2024-09-11T05:12:11.399069+0000","last_in_change":"2024-09-11T05:12:03.375435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T05:12:17.695239+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":2165098313}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":2165098313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":2165098313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":2165098313}]},"public_addr":"172.21.15.133:6808/2165098313","cluster_addr":"172.21.15.133:6809/2165098313","heartbeat_back_addr":"172.21.15.133:6811/2165098313","heartbeat_front_addr":"172.21.15.133:6810/2165098313","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":3149929203}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":3149929203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":3149929203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":3149929203}]},"public_addr":"172.21.15.133:6800/3149929203","cluster_addr":"172.21.15.133:6801/3149929203","heartbeat_back_addr":"172.21.15.133:6803/3149929203","heartbeat_front_addr":"172.21.15.133:6802/3149929203","state":["exists","up"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":841703869}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":841703869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":841703869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":841703869}]},"public_addr":"172.21.15.133:6804/841703869","cluster_addr":"172.21.15.133:6805/841703869","heartbeat_back_addr":"172.21.15.133:6807/841703869","heartbeat_front_addr":"172.21.15.133:6806/841703869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:26.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-11T05:12:17.695239+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 13, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T05:12:26.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-11T05:12:26.313 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 8 2024-09-11T05:12:26.330 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 13 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 40251081-3433-4f55-b0e4-ce0789fd19fd 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-09-11T05:12:02.234737+0000 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2024-09-11T05:12:21.467868+0000 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:crush_version 4 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:full_ratio 0.95 2024-09-11T05:12:26.605 INFO:teuthology.orchestra.run.smithi133.stdout:backfillfull_ratio 0.9 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:nearfull_ratio 0.85 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:require_min_compat_client luminous 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:min_compat_client jewel 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:require_osd_release squid 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:stretch_mode_enabled false 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout: removed_snaps_queue [2~1] 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:max_osd 3 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.133:6808/2165098313 v1:172.21.15.133:6809/2165098313 exists,up f7b6888c-0033-4334-b02d-34bc88a2cbb6 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.133:6800/3149929203 v1:172.21.15.133:6801/3149929203 exists,up 0530ffc9-4ed5-4380-929c-e39eab601f4f 2024-09-11T05:12:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.133:6804/841703869 v1:172.21.15.133:6805/841703869 exists,up 01d3910a-105e-4fa7-93fd-55d913165415 2024-09-11T05:12:26.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.133:6808/2165098313 v1:172.21.15.133:6809/2165098313 exists,up f7b6888c-0033-4334-b02d-34bc88a2cbb6', 'osd.1 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.133:6800/3149929203 v1:172.21.15.133:6801/3149929203 exists,up 0530ffc9-4ed5-4380-929c-e39eab601f4f', 'osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.133:6804/841703869 v1:172.21.15.133:6805/841703869 exists,up 01d3910a-105e-4fa7-93fd-55d913165415'] 2024-09-11T05:12:26.622 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T05:12:26.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T05:12:26.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T05:12:26.772 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:26.772 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:12:26.777 INFO:teuthology.orchestra.run.smithi133.stdout:34359738375 2024-09-11T05:12:26.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:26.783 INFO:teuthology.orchestra.run.smithi133.stdout:34359738374 2024-09-11T05:12:26.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:27.068 INFO:teuthology.orchestra.run.smithi133.stdout:34359738372 2024-09-11T05:12:27.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738372 for osd.2 2024-09-11T05:12:27.116 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:27.125 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:27.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738373 for osd.1 2024-09-11T05:12:27.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738374 got 34359738373 for osd.0 2024-09-11T05:12:28.086 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:12:28.137 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:28.144 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:28.391 INFO:teuthology.orchestra.run.smithi133.stdout:34359738373 2024-09-11T05:12:28.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738373 for osd.2 2024-09-11T05:12:28.408 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:28.434 INFO:teuthology.orchestra.run.smithi133.stdout:34359738375 2024-09-11T05:12:28.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738375 for osd.1 2024-09-11T05:12:28.451 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:28.471 INFO:teuthology.orchestra.run.smithi133.stdout:34359738374 2024-09-11T05:12:28.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738374 got 34359738374 for osd.0 2024-09-11T05:12:28.487 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:28.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-11T05:12:28.487 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:28.799 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:28.816 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-11T05:12:27.340608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81432,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034088,"statfs":{"total":289910292480,"available":289826906112,"internally_reserved":0,"allocated":352256,"data_stored":80495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.861678"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488874+0000","last_change":"2024-09-11T05:12:21.489349+0000","last_active":"2024-09-11T05:12:21.488874+0000","last_peered":"2024-09-11T05:12:21.488874+0000","last_clean":"2024-09-11T05:12:21.488874+0000","last_became_active":"2024-09-11T05:12:19.468858+0000","last_became_peered":"2024-09-11T05:12:19.468858+0000","last_unstale":"2024-09-11T05:12:21.488874+0000","last_undegraded":"2024-09-11T05:12:21.488874+0000","last_fullsized":"2024-09-11T05:12:21.488874+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:27.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488073+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:21.488073+0000","last_peered":"2024-09-11T05:12:21.488073+0000","last_clean":"2024-09-11T05:12:21.488073+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:21.488073+0000","last_undegraded":"2024-09-11T05:12:21.488073+0000","last_fullsized":"2024-09-11T05:12:21.488073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977786+0000","last_change":"2024-09-11T05:12:21.977786+0000","last_active":"2024-09-11T05:12:21.977786+0000","last_peered":"2024-09-11T05:12:21.977786+0000","last_clean":"2024-09-11T05:12:21.977786+0000","last_became_active":"2024-09-11T05:12:19.481439+0000","last_became_peered":"2024-09-11T05:12:19.481439+0000","last_unstale":"2024-09-11T05:12:21.977786+0000","last_undegraded":"2024-09-11T05:12:21.977786+0000","last_fullsized":"2024-09-11T05:12:21.977786+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:33.453711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057330499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488575+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:21.488575+0000","last_peered":"2024-09-11T05:12:21.488575+0000","last_clean":"2024-09-11T05:12:21.488575+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:21.488575+0000","last_undegraded":"2024-09-11T05:12:21.488575+0000","last_fullsized":"2024-09-11T05:12:21.488575+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487623+0000","last_change":"2024-09-11T05:12:21.488345+0000","last_active":"2024-09-11T05:12:21.487623+0000","last_peered":"2024-09-11T05:12:21.487623+0000","last_clean":"2024-09-11T05:12:21.487623+0000","last_became_active":"2024-09-11T05:12:19.468806+0000","last_became_peered":"2024-09-11T05:12:19.468806+0000","last_unstale":"2024-09-11T05:12:21.487623+0000","last_undegraded":"2024-09-11T05:12:21.487623+0000","last_fullsized":"2024-09-11T05:12:21.487623+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:47.146442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.491456+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:21.491456+0000","last_peered":"2024-09-11T05:12:21.491456+0000","last_clean":"2024-09-11T05:12:21.491456+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:21.491456+0000","last_undegraded":"2024-09-11T05:12:21.491456+0000","last_fullsized":"2024-09-11T05:12:21.491456+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977823+0000","last_change":"2024-09-11T05:12:21.977823+0000","last_active":"2024-09-11T05:12:21.977823+0000","last_peered":"2024-09-11T05:12:21.977823+0000","last_clean":"2024-09-11T05:12:21.977823+0000","last_became_active":"2024-09-11T05:12:19.477467+0000","last_became_peered":"2024-09-11T05:12:19.477467+0000","last_unstale":"2024-09-11T05:12:21.977823+0000","last_undegraded":"2024-09-11T05:12:21.977823+0000","last_fullsized":"2024-09-11T05:12:21.977823+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058685499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487499+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:21.487499+0000","last_peered":"2024-09-11T05:12:21.487499+0000","last_clean":"2024-09-11T05:12:21.487499+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:21.487499+0000","last_undegraded":"2024-09-11T05:12:21.487499+0000","last_fullsized":"2024-09-11T05:12:21.487499+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":114688,"data_stored":26819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:28.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:29.092 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:29.093 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:29.108 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-11T05:12:27.340608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81432,"kb_used_data":344,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034088,"statfs":{"total":289910292480,"available":289826906112,"internally_reserved":0,"allocated":352256,"data_stored":80495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.861678"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488874+0000","last_change":"2024-09-11T05:12:21.489349+0000","last_active":"2024-09-11T05:12:21.488874+0000","last_peered":"2024-09-11T05:12:21.488874+0000","last_clean":"2024-09-11T05:12:21.488874+0000","last_became_active":"2024-09-11T05:12:19.468858+0000","last_became_peered":"2024-09-11T05:12:19.468858+0000","last_unstale":"2024-09-11T05:12:21.488874+0000","last_undegraded":"2024-09-11T05:12:21.488874+0000","last_fullsized":"2024-09-11T05:12:21.488874+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:27.648705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488073+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:21.488073+0000","last_peered":"2024-09-11T05:12:21.488073+0000","last_clean":"2024-09-11T05:12:21.488073+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:21.488073+0000","last_undegraded":"2024-09-11T05:12:21.488073+0000","last_fullsized":"2024-09-11T05:12:21.488073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977786+0000","last_change":"2024-09-11T05:12:21.977786+0000","last_active":"2024-09-11T05:12:21.977786+0000","last_peered":"2024-09-11T05:12:21.977786+0000","last_clean":"2024-09-11T05:12:21.977786+0000","last_became_active":"2024-09-11T05:12:19.481439+0000","last_became_peered":"2024-09-11T05:12:19.481439+0000","last_unstale":"2024-09-11T05:12:21.977786+0000","last_undegraded":"2024-09-11T05:12:21.977786+0000","last_fullsized":"2024-09-11T05:12:21.977786+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:33.453711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057330499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.488575+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:21.488575+0000","last_peered":"2024-09-11T05:12:21.488575+0000","last_clean":"2024-09-11T05:12:21.488575+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:21.488575+0000","last_undegraded":"2024-09-11T05:12:21.488575+0000","last_fullsized":"2024-09-11T05:12:21.488575+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487623+0000","last_change":"2024-09-11T05:12:21.488345+0000","last_active":"2024-09-11T05:12:21.487623+0000","last_peered":"2024-09-11T05:12:21.487623+0000","last_clean":"2024-09-11T05:12:21.487623+0000","last_became_active":"2024-09-11T05:12:19.468806+0000","last_became_peered":"2024-09-11T05:12:19.468806+0000","last_unstale":"2024-09-11T05:12:21.487623+0000","last_undegraded":"2024-09-11T05:12:21.487623+0000","last_fullsized":"2024-09-11T05:12:21.487623+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:47.146442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.491456+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:21.491456+0000","last_peered":"2024-09-11T05:12:21.491456+0000","last_clean":"2024-09-11T05:12:21.491456+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:21.491456+0000","last_undegraded":"2024-09-11T05:12:21.491456+0000","last_fullsized":"2024-09-11T05:12:21.491456+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.977823+0000","last_change":"2024-09-11T05:12:21.977823+0000","last_active":"2024-09-11T05:12:21.977823+0000","last_peered":"2024-09-11T05:12:21.977823+0000","last_clean":"2024-09-11T05:12:21.977823+0000","last_became_active":"2024-09-11T05:12:19.477467+0000","last_became_peered":"2024-09-11T05:12:19.477467+0000","last_unstale":"2024-09-11T05:12:21.977823+0000","last_undegraded":"2024-09-11T05:12:21.977823+0000","last_fullsized":"2024-09-11T05:12:21.977823+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058685499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-11T05:12:21.487499+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:21.487499+0000","last_peered":"2024-09-11T05:12:21.487499+0000","last_clean":"2024-09-11T05:12:21.487499+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:21.487499+0000","last_undegraded":"2024-09-11T05:12:21.487499+0000","last_fullsized":"2024-09-11T05:12:21.487499+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":114688,"data_stored":26819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":118784,"data_stored":26838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:29.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-11T05:12:29.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-09-11T05:12:29.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-09-11T05:12:30.135 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'unfounddel_pool' created 2024-09-11T05:12:30.165 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-09-11T05:12:31.155 INFO:teuthology.orchestra.run.smithi133.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-09-11T05:12:32.194 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T05:12:32.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:12:32.256 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T05:12:32.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-11T05:12:32.572 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:12:32.568+0000 7f9b75702640 -1 mon.a@0(leader).osd e17 definitely_dead 0 2024-09-11T05:12:33.165 INFO:teuthology.orchestra.run.smithi133.stderr:marked down osd.2. 2024-09-11T05:12:33.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-11T05:12:33.505 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:12:33.504+0000 7f9b75702640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-09-11T05:12:34.178 INFO:teuthology.orchestra.run.smithi133.stderr:marked out osd.2. 2024-09-11T05:12:34.205 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-09-11T05:12:34.205 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-09-11T05:12:34.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T05:12:34.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T05:12:34.467 INFO:teuthology.orchestra.run.smithi133.stdout:34359738377 2024-09-11T05:12:34.467 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:34.468 INFO:teuthology.orchestra.run.smithi133.stdout:34359738377 2024-09-11T05:12:34.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:34.773 INFO:teuthology.orchestra.run.smithi133.stdout:34359738376 2024-09-11T05:12:34.790 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.0 2024-09-11T05:12:34.808 INFO:teuthology.orchestra.run.smithi133.stdout:34359738376 2024-09-11T05:12:34.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-11T05:12:35.791 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:35.826 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:36.096 INFO:teuthology.orchestra.run.smithi133.stdout:34359738376 2024-09-11T05:12:36.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.0 2024-09-11T05:12:36.141 INFO:teuthology.orchestra.run.smithi133.stdout:34359738376 2024-09-11T05:12:36.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-11T05:12:37.114 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:37.160 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:37.455 INFO:teuthology.orchestra.run.smithi133.stdout:34359738377 2024-09-11T05:12:37.474 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738377 for osd.0 2024-09-11T05:12:37.474 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:37.483 INFO:teuthology.orchestra.run.smithi133.stdout:34359738378 2024-09-11T05:12:37.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738378 for osd.1 2024-09-11T05:12:37.501 DEBUG:teuthology.parallel:result is None 2024-09-11T05:12:37.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-11T05:12:37.502 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:37.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:37.858 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-11T05:12:37.342953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54388,"kb_used_data":332,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689292,"statfs":{"total":193273528320,"available":193217835008,"internally_reserved":0,"allocated":339968,"data_stored":88923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.183683"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.190564+0000","last_change":"2024-09-11T05:12:31.174556+0000","last_active":"2024-09-11T05:12:34.190564+0000","last_peered":"2024-09-11T05:12:34.190564+0000","last_clean":"2024-09-11T05:12:34.190564+0000","last_became_active":"2024-09-11T05:12:31.172693+0000","last_became_peered":"2024-09-11T05:12:31.172693+0000","last_unstale":"2024-09-11T05:12:34.190564+0000","last_undegraded":"2024-09-11T05:12:34.190564+0000","last_fullsized":"2024-09-11T05:12:34.190564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:12.712402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190344+0000","last_change":"2024-09-11T05:12:33.175623+0000","last_active":"2024-09-11T05:12:30.129133+0000","last_peered":"2024-09-11T05:12:30.129133+0000","last_clean":"2024-09-11T05:12:30.129133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190344+0000","last_undegraded":"2024-09-11T05:12:34.190344+0000","last_fullsized":"2024-09-11T05:12:34.190344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209515+0000","last_change":"2024-09-11T05:12:35.208981+0000","last_active":"2024-09-11T05:12:35.209515+0000","last_peered":"2024-09-11T05:12:35.209515+0000","last_clean":"2024-09-11T05:12:35.209515+0000","last_became_active":"2024-09-11T05:12:35.208030+0000","last_became_peered":"2024-09-11T05:12:35.208030+0000","last_unstale":"2024-09-11T05:12:35.209515+0000","last_undegraded":"2024-09-11T05:12:35.209515+0000","last_fullsized":"2024-09-11T05:12:35.209515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:35.405376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190463+0000","last_change":"2024-09-11T05:12:33.175347+0000","last_active":"2024-09-11T05:12:30.129133+0000","last_peered":"2024-09-11T05:12:30.129133+0000","last_clean":"2024-09-11T05:12:30.129133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190463+0000","last_undegraded":"2024-09-11T05:12:34.190463+0000","last_fullsized":"2024-09-11T05:12:34.190463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-11T05:12:33.175276+0000","last_change":"2024-09-11T05:12:33.175276+0000","last_active":"2024-09-11T05:12:33.175276+0000","last_peered":"2024-09-11T05:12:33.174627+0000","last_clean":"2024-09-11T05:12:33.174627+0000","last_became_active":"2024-09-11T05:12:31.174649+0000","last_became_peered":"2024-09-11T05:12:31.174649+0000","last_unstale":"2024-09-11T05:12:33.175276+0000","last_undegraded":"2024-09-11T05:12:33.175276+0000","last_fullsized":"2024-09-11T05:12:33.175276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195617+0000","last_change":"2024-09-11T05:12:31.176835+0000","last_active":"2024-09-11T05:12:35.195617+0000","last_peered":"2024-09-11T05:12:35.195617+0000","last_clean":"2024-09-11T05:12:35.195617+0000","last_became_active":"2024-09-11T05:12:31.176545+0000","last_became_peered":"2024-09-11T05:12:31.176545+0000","last_unstale":"2024-09-11T05:12:35.195617+0000","last_undegraded":"2024-09-11T05:12:35.195617+0000","last_fullsized":"2024-09-11T05:12:35.195617+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:45.064646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195994+0000","last_change":"2024-09-11T05:12:31.170829+0000","last_active":"2024-09-11T05:12:35.195994+0000","last_peered":"2024-09-11T05:12:35.195994+0000","last_clean":"2024-09-11T05:12:35.195994+0000","last_became_active":"2024-09-11T05:12:31.170224+0000","last_became_peered":"2024-09-11T05:12:31.170224+0000","last_unstale":"2024-09-11T05:12:35.195994+0000","last_undegraded":"2024-09-11T05:12:35.195994+0000","last_fullsized":"2024-09-11T05:12:35.195994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:42.377273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-11T05:12:33.174552+0000","last_change":"2024-09-11T05:12:33.174552+0000","last_active":"2024-09-11T05:12:33.174552+0000","last_peered":"2024-09-11T05:12:33.173961+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:12:31.169197+0000","last_became_peered":"2024-09-11T05:12:31.169197+0000","last_unstale":"2024-09-11T05:12:33.174552+0000","last_undegraded":"2024-09-11T05:12:33.174552+0000","last_fullsized":"2024-09-11T05:12:33.174552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.196633+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:35.196633+0000","last_peered":"2024-09-11T05:12:35.196633+0000","last_clean":"2024-09-11T05:12:35.196633+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:35.196633+0000","last_undegraded":"2024-09-11T05:12:35.196633+0000","last_fullsized":"2024-09-11T05:12:35.196633+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209280+0000","last_change":"2024-09-11T05:12:35.209014+0000","last_active":"2024-09-11T05:12:35.209280+0000","last_peered":"2024-09-11T05:12:35.209280+0000","last_clean":"2024-09-11T05:12:35.209280+0000","last_became_active":"2024-09-11T05:12:35.208141+0000","last_became_peered":"2024-09-11T05:12:35.208141+0000","last_unstale":"2024-09-11T05:12:35.209280+0000","last_undegraded":"2024-09-11T05:12:35.209280+0000","last_fullsized":"2024-09-11T05:12:35.209280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:47.208445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.192880+0000","last_change":"2024-09-11T05:12:33.174670+0000","last_active":"2024-09-11T05:12:18.438492+0000","last_peered":"2024-09-11T05:12:18.438492+0000","last_clean":"2024-09-11T05:12:18.438492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.192880+0000","last_undegraded":"2024-09-11T05:12:34.192880+0000","last_fullsized":"2024-09-11T05:12:34.192880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.189843+0000","last_change":"2024-09-11T05:12:31.177431+0000","last_active":"2024-09-11T05:12:34.189843+0000","last_peered":"2024-09-11T05:12:34.189843+0000","last_clean":"2024-09-11T05:12:34.189843+0000","last_became_active":"2024-09-11T05:12:31.175358+0000","last_became_peered":"2024-09-11T05:12:31.175358+0000","last_unstale":"2024-09-11T05:12:34.189843+0000","last_undegraded":"2024-09-11T05:12:34.189843+0000","last_fullsized":"2024-09-11T05:12:34.189843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:07.430680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":33,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.189919+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:34.189919+0000","last_peered":"2024-09-11T05:12:34.189919+0000","last_clean":"2024-09-11T05:12:34.189919+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:34.189919+0000","last_undegraded":"2024-09-11T05:12:34.189919+0000","last_fullsized":"2024-09-11T05:12:34.189919+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.205228+0000","last_change":"2024-09-11T05:12:35.205045+0000","last_active":"2024-09-11T05:12:35.205228+0000","last_peered":"2024-09-11T05:12:35.205228+0000","last_clean":"2024-09-11T05:12:35.205228+0000","last_became_active":"2024-09-11T05:12:35.204315+0000","last_became_peered":"2024-09-11T05:12:35.204315+0000","last_unstale":"2024-09-11T05:12:35.205228+0000","last_undegraded":"2024-09-11T05:12:35.205228+0000","last_fullsized":"2024-09-11T05:12:35.205228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:01.399867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":44,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.216831+0000","last_change":"2024-09-11T05:12:35.216562+0000","last_active":"2024-09-11T05:12:35.216831+0000","last_peered":"2024-09-11T05:12:35.216831+0000","last_clean":"2024-09-11T05:12:35.216831+0000","last_became_active":"2024-09-11T05:12:35.211123+0000","last_became_peered":"2024-09-11T05:12:35.211123+0000","last_unstale":"2024-09-11T05:12:35.216831+0000","last_undegraded":"2024-09-11T05:12:35.216831+0000","last_fullsized":"2024-09-11T05:12:35.216831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:35.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.202696+0000","last_change":"2024-09-11T05:12:35.202448+0000","last_active":"2024-09-11T05:12:35.202696+0000","last_peered":"2024-09-11T05:12:35.202696+0000","last_clean":"2024-09-11T05:12:35.202696+0000","last_became_active":"2024-09-11T05:12:35.200425+0000","last_became_peered":"2024-09-11T05:12:35.200425+0000","last_unstale":"2024-09-11T05:12:35.202696+0000","last_undegraded":"2024-09-11T05:12:35.202696+0000","last_fullsized":"2024-09-11T05:12:35.202696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:11.836522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197894+0000","last_change":"2024-09-11T05:12:31.170669+0000","last_active":"2024-09-11T05:12:35.197894+0000","last_peered":"2024-09-11T05:12:35.197894+0000","last_clean":"2024-09-11T05:12:35.197894+0000","last_became_active":"2024-09-11T05:12:31.170245+0000","last_became_peered":"2024-09-11T05:12:31.170245+0000","last_unstale":"2024-09-11T05:12:35.197894+0000","last_undegraded":"2024-09-11T05:12:35.197894+0000","last_fullsized":"2024-09-11T05:12:35.197894+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:41.806823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195910+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:35.195910+0000","last_peered":"2024-09-11T05:12:35.195910+0000","last_clean":"2024-09-11T05:12:35.195910+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:35.195910+0000","last_undegraded":"2024-09-11T05:12:35.195910+0000","last_fullsized":"2024-09-11T05:12:35.195910+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198180+0000","last_change":"2024-09-11T05:12:31.175435+0000","last_active":"2024-09-11T05:12:35.198180+0000","last_peered":"2024-09-11T05:12:35.198180+0000","last_clean":"2024-09-11T05:12:35.198180+0000","last_became_active":"2024-09-11T05:12:31.174826+0000","last_became_peered":"2024-09-11T05:12:31.174826+0000","last_unstale":"2024-09-11T05:12:35.198180+0000","last_undegraded":"2024-09-11T05:12:35.198180+0000","last_fullsized":"2024-09-11T05:12:35.198180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:26.974983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190058+0000","last_change":"2024-09-11T05:12:33.176083+0000","last_active":"2024-09-11T05:12:18.438492+0000","last_peered":"2024-09-11T05:12:18.438492+0000","last_clean":"2024-09-11T05:12:18.438492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190058+0000","last_undegraded":"2024-09-11T05:12:34.190058+0000","last_fullsized":"2024-09-11T05:12:34.190058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197897+0000","last_change":"2024-09-11T05:12:31.173875+0000","last_active":"2024-09-11T05:12:35.197897+0000","last_peered":"2024-09-11T05:12:35.197897+0000","last_clean":"2024-09-11T05:12:35.197897+0000","last_became_active":"2024-09-11T05:12:31.173368+0000","last_became_peered":"2024-09-11T05:12:31.173368+0000","last_unstale":"2024-09-11T05:12:35.197897+0000","last_undegraded":"2024-09-11T05:12:35.197897+0000","last_fullsized":"2024-09-11T05:12:35.197897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:00.201066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198313+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:35.198313+0000","last_peered":"2024-09-11T05:12:35.198313+0000","last_clean":"2024-09-11T05:12:35.198313+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:35.198313+0000","last_undegraded":"2024-09-11T05:12:35.198313+0000","last_fullsized":"2024-09-11T05:12:35.198313+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209255+0000","last_change":"2024-09-11T05:12:35.208978+0000","last_active":"2024-09-11T05:12:35.209255+0000","last_peered":"2024-09-11T05:12:35.209255+0000","last_clean":"2024-09-11T05:12:35.209255+0000","last_became_active":"2024-09-11T05:12:35.208254+0000","last_became_peered":"2024-09-11T05:12:35.208254+0000","last_unstale":"2024-09-11T05:12:35.209255+0000","last_undegraded":"2024-09-11T05:12:35.209255+0000","last_fullsized":"2024-09-11T05:12:35.209255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:32.748794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"19'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195517+0000","last_change":"2024-09-11T05:12:31.174174+0000","last_active":"2024-09-11T05:12:35.195517+0000","last_peered":"2024-09-11T05:12:35.195517+0000","last_clean":"2024-09-11T05:12:35.195517+0000","last_became_active":"2024-09-11T05:12:31.173727+0000","last_became_peered":"2024-09-11T05:12:31.173727+0000","last_unstale":"2024-09-11T05:12:35.195517+0000","last_undegraded":"2024-09-11T05:12:35.195517+0000","last_fullsized":"2024-09-11T05:12:35.195517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:58.026864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":47516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":41407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:37.860 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:38.151 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:38.152 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:38.167 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-11T05:12:37.342953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54388,"kb_used_data":332,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689292,"statfs":{"total":193273528320,"available":193217835008,"internally_reserved":0,"allocated":339968,"data_stored":88923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.183683"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.190564+0000","last_change":"2024-09-11T05:12:31.174556+0000","last_active":"2024-09-11T05:12:34.190564+0000","last_peered":"2024-09-11T05:12:34.190564+0000","last_clean":"2024-09-11T05:12:34.190564+0000","last_became_active":"2024-09-11T05:12:31.172693+0000","last_became_peered":"2024-09-11T05:12:31.172693+0000","last_unstale":"2024-09-11T05:12:34.190564+0000","last_undegraded":"2024-09-11T05:12:34.190564+0000","last_fullsized":"2024-09-11T05:12:34.190564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:12.712402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190344+0000","last_change":"2024-09-11T05:12:33.175623+0000","last_active":"2024-09-11T05:12:30.129133+0000","last_peered":"2024-09-11T05:12:30.129133+0000","last_clean":"2024-09-11T05:12:30.129133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190344+0000","last_undegraded":"2024-09-11T05:12:34.190344+0000","last_fullsized":"2024-09-11T05:12:34.190344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209515+0000","last_change":"2024-09-11T05:12:35.208981+0000","last_active":"2024-09-11T05:12:35.209515+0000","last_peered":"2024-09-11T05:12:35.209515+0000","last_clean":"2024-09-11T05:12:35.209515+0000","last_became_active":"2024-09-11T05:12:35.208030+0000","last_became_peered":"2024-09-11T05:12:35.208030+0000","last_unstale":"2024-09-11T05:12:35.209515+0000","last_undegraded":"2024-09-11T05:12:35.209515+0000","last_fullsized":"2024-09-11T05:12:35.209515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:35.405376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190463+0000","last_change":"2024-09-11T05:12:33.175347+0000","last_active":"2024-09-11T05:12:30.129133+0000","last_peered":"2024-09-11T05:12:30.129133+0000","last_clean":"2024-09-11T05:12:30.129133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190463+0000","last_undegraded":"2024-09-11T05:12:34.190463+0000","last_fullsized":"2024-09-11T05:12:34.190463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-11T05:12:33.175276+0000","last_change":"2024-09-11T05:12:33.175276+0000","last_active":"2024-09-11T05:12:33.175276+0000","last_peered":"2024-09-11T05:12:33.174627+0000","last_clean":"2024-09-11T05:12:33.174627+0000","last_became_active":"2024-09-11T05:12:31.174649+0000","last_became_peered":"2024-09-11T05:12:31.174649+0000","last_unstale":"2024-09-11T05:12:33.175276+0000","last_undegraded":"2024-09-11T05:12:33.175276+0000","last_fullsized":"2024-09-11T05:12:33.175276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195617+0000","last_change":"2024-09-11T05:12:31.176835+0000","last_active":"2024-09-11T05:12:35.195617+0000","last_peered":"2024-09-11T05:12:35.195617+0000","last_clean":"2024-09-11T05:12:35.195617+0000","last_became_active":"2024-09-11T05:12:31.176545+0000","last_became_peered":"2024-09-11T05:12:31.176545+0000","last_unstale":"2024-09-11T05:12:35.195617+0000","last_undegraded":"2024-09-11T05:12:35.195617+0000","last_fullsized":"2024-09-11T05:12:35.195617+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:45.064646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195994+0000","last_change":"2024-09-11T05:12:31.170829+0000","last_active":"2024-09-11T05:12:35.195994+0000","last_peered":"2024-09-11T05:12:35.195994+0000","last_clean":"2024-09-11T05:12:35.195994+0000","last_became_active":"2024-09-11T05:12:31.170224+0000","last_became_peered":"2024-09-11T05:12:31.170224+0000","last_unstale":"2024-09-11T05:12:35.195994+0000","last_undegraded":"2024-09-11T05:12:35.195994+0000","last_fullsized":"2024-09-11T05:12:35.195994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:42.377273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-11T05:12:33.174552+0000","last_change":"2024-09-11T05:12:33.174552+0000","last_active":"2024-09-11T05:12:33.174552+0000","last_peered":"2024-09-11T05:12:33.173961+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:12:31.169197+0000","last_became_peered":"2024-09-11T05:12:31.169197+0000","last_unstale":"2024-09-11T05:12:33.174552+0000","last_undegraded":"2024-09-11T05:12:33.174552+0000","last_fullsized":"2024-09-11T05:12:33.174552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.196633+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:35.196633+0000","last_peered":"2024-09-11T05:12:35.196633+0000","last_clean":"2024-09-11T05:12:35.196633+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:35.196633+0000","last_undegraded":"2024-09-11T05:12:35.196633+0000","last_fullsized":"2024-09-11T05:12:35.196633+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209280+0000","last_change":"2024-09-11T05:12:35.209014+0000","last_active":"2024-09-11T05:12:35.209280+0000","last_peered":"2024-09-11T05:12:35.209280+0000","last_clean":"2024-09-11T05:12:35.209280+0000","last_became_active":"2024-09-11T05:12:35.208141+0000","last_became_peered":"2024-09-11T05:12:35.208141+0000","last_unstale":"2024-09-11T05:12:35.209280+0000","last_undegraded":"2024-09-11T05:12:35.209280+0000","last_fullsized":"2024-09-11T05:12:35.209280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:47.208445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.192880+0000","last_change":"2024-09-11T05:12:33.174670+0000","last_active":"2024-09-11T05:12:18.438492+0000","last_peered":"2024-09-11T05:12:18.438492+0000","last_clean":"2024-09-11T05:12:18.438492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.192880+0000","last_undegraded":"2024-09-11T05:12:34.192880+0000","last_fullsized":"2024-09-11T05:12:34.192880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.189843+0000","last_change":"2024-09-11T05:12:31.177431+0000","last_active":"2024-09-11T05:12:34.189843+0000","last_peered":"2024-09-11T05:12:34.189843+0000","last_clean":"2024-09-11T05:12:34.189843+0000","last_became_active":"2024-09-11T05:12:31.175358+0000","last_became_peered":"2024-09-11T05:12:31.175358+0000","last_unstale":"2024-09-11T05:12:34.189843+0000","last_undegraded":"2024-09-11T05:12:34.189843+0000","last_fullsized":"2024-09-11T05:12:34.189843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:07.430680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":33,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.189919+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:34.189919+0000","last_peered":"2024-09-11T05:12:34.189919+0000","last_clean":"2024-09-11T05:12:34.189919+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:34.189919+0000","last_undegraded":"2024-09-11T05:12:34.189919+0000","last_fullsized":"2024-09-11T05:12:34.189919+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.205228+0000","last_change":"2024-09-11T05:12:35.205045+0000","last_active":"2024-09-11T05:12:35.205228+0000","last_peered":"2024-09-11T05:12:35.205228+0000","last_clean":"2024-09-11T05:12:35.205228+0000","last_became_active":"2024-09-11T05:12:35.204315+0000","last_became_peered":"2024-09-11T05:12:35.204315+0000","last_unstale":"2024-09-11T05:12:35.205228+0000","last_undegraded":"2024-09-11T05:12:35.205228+0000","last_fullsized":"2024-09-11T05:12:35.205228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:01.399867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":44,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.216831+0000","last_change":"2024-09-11T05:12:35.216562+0000","last_active":"2024-09-11T05:12:35.216831+0000","last_peered":"2024-09-11T05:12:35.216831+0000","last_clean":"2024-09-11T05:12:35.216831+0000","last_became_active":"2024-09-11T05:12:35.211123+0000","last_became_peered":"2024-09-11T05:12:35.211123+0000","last_unstale":"2024-09-11T05:12:35.216831+0000","last_undegraded":"2024-09-11T05:12:35.216831+0000","last_fullsized":"2024-09-11T05:12:35.216831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:35.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.202696+0000","last_change":"2024-09-11T05:12:35.202448+0000","last_active":"2024-09-11T05:12:35.202696+0000","last_peered":"2024-09-11T05:12:35.202696+0000","last_clean":"2024-09-11T05:12:35.202696+0000","last_became_active":"2024-09-11T05:12:35.200425+0000","last_became_peered":"2024-09-11T05:12:35.200425+0000","last_unstale":"2024-09-11T05:12:35.202696+0000","last_undegraded":"2024-09-11T05:12:35.202696+0000","last_fullsized":"2024-09-11T05:12:35.202696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:11.836522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197894+0000","last_change":"2024-09-11T05:12:31.170669+0000","last_active":"2024-09-11T05:12:35.197894+0000","last_peered":"2024-09-11T05:12:35.197894+0000","last_clean":"2024-09-11T05:12:35.197894+0000","last_became_active":"2024-09-11T05:12:31.170245+0000","last_became_peered":"2024-09-11T05:12:31.170245+0000","last_unstale":"2024-09-11T05:12:35.197894+0000","last_undegraded":"2024-09-11T05:12:35.197894+0000","last_fullsized":"2024-09-11T05:12:35.197894+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:41.806823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195910+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:35.195910+0000","last_peered":"2024-09-11T05:12:35.195910+0000","last_clean":"2024-09-11T05:12:35.195910+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:35.195910+0000","last_undegraded":"2024-09-11T05:12:35.195910+0000","last_fullsized":"2024-09-11T05:12:35.195910+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198180+0000","last_change":"2024-09-11T05:12:31.175435+0000","last_active":"2024-09-11T05:12:35.198180+0000","last_peered":"2024-09-11T05:12:35.198180+0000","last_clean":"2024-09-11T05:12:35.198180+0000","last_became_active":"2024-09-11T05:12:31.174826+0000","last_became_peered":"2024-09-11T05:12:31.174826+0000","last_unstale":"2024-09-11T05:12:35.198180+0000","last_undegraded":"2024-09-11T05:12:35.198180+0000","last_fullsized":"2024-09-11T05:12:35.198180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:26.974983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190058+0000","last_change":"2024-09-11T05:12:33.176083+0000","last_active":"2024-09-11T05:12:18.438492+0000","last_peered":"2024-09-11T05:12:18.438492+0000","last_clean":"2024-09-11T05:12:18.438492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190058+0000","last_undegraded":"2024-09-11T05:12:34.190058+0000","last_fullsized":"2024-09-11T05:12:34.190058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197897+0000","last_change":"2024-09-11T05:12:31.173875+0000","last_active":"2024-09-11T05:12:35.197897+0000","last_peered":"2024-09-11T05:12:35.197897+0000","last_clean":"2024-09-11T05:12:35.197897+0000","last_became_active":"2024-09-11T05:12:31.173368+0000","last_became_peered":"2024-09-11T05:12:31.173368+0000","last_unstale":"2024-09-11T05:12:35.197897+0000","last_undegraded":"2024-09-11T05:12:35.197897+0000","last_fullsized":"2024-09-11T05:12:35.197897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:00.201066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198313+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:35.198313+0000","last_peered":"2024-09-11T05:12:35.198313+0000","last_clean":"2024-09-11T05:12:35.198313+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:35.198313+0000","last_undegraded":"2024-09-11T05:12:35.198313+0000","last_fullsized":"2024-09-11T05:12:35.198313+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209255+0000","last_change":"2024-09-11T05:12:35.208978+0000","last_active":"2024-09-11T05:12:35.209255+0000","last_peered":"2024-09-11T05:12:35.209255+0000","last_clean":"2024-09-11T05:12:35.209255+0000","last_became_active":"2024-09-11T05:12:35.208254+0000","last_became_peered":"2024-09-11T05:12:35.208254+0000","last_unstale":"2024-09-11T05:12:35.209255+0000","last_undegraded":"2024-09-11T05:12:35.209255+0000","last_fullsized":"2024-09-11T05:12:35.209255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:32.748794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"19'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195517+0000","last_change":"2024-09-11T05:12:31.174174+0000","last_active":"2024-09-11T05:12:35.195517+0000","last_peered":"2024-09-11T05:12:35.195517+0000","last_clean":"2024-09-11T05:12:35.195517+0000","last_became_active":"2024-09-11T05:12:31.173727+0000","last_became_peered":"2024-09-11T05:12:31.173727+0000","last_unstale":"2024-09-11T05:12:35.195517+0000","last_undegraded":"2024-09-11T05:12:35.195517+0000","last_fullsized":"2024-09-11T05:12:35.195517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:58.026864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":47516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":41407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:38.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:38.451 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:38.467 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-11T05:12:37.342953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54388,"kb_used_data":332,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689292,"statfs":{"total":193273528320,"available":193217835008,"internally_reserved":0,"allocated":339968,"data_stored":88923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.183683"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.190564+0000","last_change":"2024-09-11T05:12:31.174556+0000","last_active":"2024-09-11T05:12:34.190564+0000","last_peered":"2024-09-11T05:12:34.190564+0000","last_clean":"2024-09-11T05:12:34.190564+0000","last_became_active":"2024-09-11T05:12:31.172693+0000","last_became_peered":"2024-09-11T05:12:31.172693+0000","last_unstale":"2024-09-11T05:12:34.190564+0000","last_undegraded":"2024-09-11T05:12:34.190564+0000","last_fullsized":"2024-09-11T05:12:34.190564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:12.712402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190344+0000","last_change":"2024-09-11T05:12:33.175623+0000","last_active":"2024-09-11T05:12:30.129133+0000","last_peered":"2024-09-11T05:12:30.129133+0000","last_clean":"2024-09-11T05:12:30.129133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190344+0000","last_undegraded":"2024-09-11T05:12:34.190344+0000","last_fullsized":"2024-09-11T05:12:34.190344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209515+0000","last_change":"2024-09-11T05:12:35.208981+0000","last_active":"2024-09-11T05:12:35.209515+0000","last_peered":"2024-09-11T05:12:35.209515+0000","last_clean":"2024-09-11T05:12:35.209515+0000","last_became_active":"2024-09-11T05:12:35.208030+0000","last_became_peered":"2024-09-11T05:12:35.208030+0000","last_unstale":"2024-09-11T05:12:35.209515+0000","last_undegraded":"2024-09-11T05:12:35.209515+0000","last_fullsized":"2024-09-11T05:12:35.209515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:35.405376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190463+0000","last_change":"2024-09-11T05:12:33.175347+0000","last_active":"2024-09-11T05:12:30.129133+0000","last_peered":"2024-09-11T05:12:30.129133+0000","last_clean":"2024-09-11T05:12:30.129133+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190463+0000","last_undegraded":"2024-09-11T05:12:34.190463+0000","last_fullsized":"2024-09-11T05:12:34.190463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-11T05:12:33.175276+0000","last_change":"2024-09-11T05:12:33.175276+0000","last_active":"2024-09-11T05:12:33.175276+0000","last_peered":"2024-09-11T05:12:33.174627+0000","last_clean":"2024-09-11T05:12:33.174627+0000","last_became_active":"2024-09-11T05:12:31.174649+0000","last_became_peered":"2024-09-11T05:12:31.174649+0000","last_unstale":"2024-09-11T05:12:33.175276+0000","last_undegraded":"2024-09-11T05:12:33.175276+0000","last_fullsized":"2024-09-11T05:12:33.175276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195617+0000","last_change":"2024-09-11T05:12:31.176835+0000","last_active":"2024-09-11T05:12:35.195617+0000","last_peered":"2024-09-11T05:12:35.195617+0000","last_clean":"2024-09-11T05:12:35.195617+0000","last_became_active":"2024-09-11T05:12:31.176545+0000","last_became_peered":"2024-09-11T05:12:31.176545+0000","last_unstale":"2024-09-11T05:12:35.195617+0000","last_undegraded":"2024-09-11T05:12:35.195617+0000","last_fullsized":"2024-09-11T05:12:35.195617+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:45.064646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195994+0000","last_change":"2024-09-11T05:12:31.170829+0000","last_active":"2024-09-11T05:12:35.195994+0000","last_peered":"2024-09-11T05:12:35.195994+0000","last_clean":"2024-09-11T05:12:35.195994+0000","last_became_active":"2024-09-11T05:12:31.170224+0000","last_became_peered":"2024-09-11T05:12:31.170224+0000","last_unstale":"2024-09-11T05:12:35.195994+0000","last_undegraded":"2024-09-11T05:12:35.195994+0000","last_fullsized":"2024-09-11T05:12:35.195994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:42.377273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"peering","last_fresh":"2024-09-11T05:12:33.174552+0000","last_change":"2024-09-11T05:12:33.174552+0000","last_active":"2024-09-11T05:12:33.174552+0000","last_peered":"2024-09-11T05:12:33.173961+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:12:31.169197+0000","last_became_peered":"2024-09-11T05:12:31.169197+0000","last_unstale":"2024-09-11T05:12:33.174552+0000","last_undegraded":"2024-09-11T05:12:33.174552+0000","last_fullsized":"2024-09-11T05:12:33.174552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.196633+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:35.196633+0000","last_peered":"2024-09-11T05:12:35.196633+0000","last_clean":"2024-09-11T05:12:35.196633+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:35.196633+0000","last_undegraded":"2024-09-11T05:12:35.196633+0000","last_fullsized":"2024-09-11T05:12:35.196633+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209280+0000","last_change":"2024-09-11T05:12:35.209014+0000","last_active":"2024-09-11T05:12:35.209280+0000","last_peered":"2024-09-11T05:12:35.209280+0000","last_clean":"2024-09-11T05:12:35.209280+0000","last_became_active":"2024-09-11T05:12:35.208141+0000","last_became_peered":"2024-09-11T05:12:35.208141+0000","last_unstale":"2024-09-11T05:12:35.209280+0000","last_undegraded":"2024-09-11T05:12:35.209280+0000","last_fullsized":"2024-09-11T05:12:35.209280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:47.208445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.192880+0000","last_change":"2024-09-11T05:12:33.174670+0000","last_active":"2024-09-11T05:12:18.438492+0000","last_peered":"2024-09-11T05:12:18.438492+0000","last_clean":"2024-09-11T05:12:18.438492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.192880+0000","last_undegraded":"2024-09-11T05:12:34.192880+0000","last_fullsized":"2024-09-11T05:12:34.192880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.189843+0000","last_change":"2024-09-11T05:12:31.177431+0000","last_active":"2024-09-11T05:12:34.189843+0000","last_peered":"2024-09-11T05:12:34.189843+0000","last_clean":"2024-09-11T05:12:34.189843+0000","last_became_active":"2024-09-11T05:12:31.175358+0000","last_became_peered":"2024-09-11T05:12:31.175358+0000","last_unstale":"2024-09-11T05:12:34.189843+0000","last_undegraded":"2024-09-11T05:12:34.189843+0000","last_fullsized":"2024-09-11T05:12:34.189843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:07.430680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":33,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T05:12:34.189919+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:34.189919+0000","last_peered":"2024-09-11T05:12:34.189919+0000","last_clean":"2024-09-11T05:12:34.189919+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:34.189919+0000","last_undegraded":"2024-09-11T05:12:34.189919+0000","last_fullsized":"2024-09-11T05:12:34.189919+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.205228+0000","last_change":"2024-09-11T05:12:35.205045+0000","last_active":"2024-09-11T05:12:35.205228+0000","last_peered":"2024-09-11T05:12:35.205228+0000","last_clean":"2024-09-11T05:12:35.205228+0000","last_became_active":"2024-09-11T05:12:35.204315+0000","last_became_peered":"2024-09-11T05:12:35.204315+0000","last_unstale":"2024-09-11T05:12:35.205228+0000","last_undegraded":"2024-09-11T05:12:35.205228+0000","last_fullsized":"2024-09-11T05:12:35.205228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:01.399867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":44,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.216831+0000","last_change":"2024-09-11T05:12:35.216562+0000","last_active":"2024-09-11T05:12:35.216831+0000","last_peered":"2024-09-11T05:12:35.216831+0000","last_clean":"2024-09-11T05:12:35.216831+0000","last_became_active":"2024-09-11T05:12:35.211123+0000","last_became_peered":"2024-09-11T05:12:35.211123+0000","last_unstale":"2024-09-11T05:12:35.216831+0000","last_undegraded":"2024-09-11T05:12:35.216831+0000","last_fullsized":"2024-09-11T05:12:35.216831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:35.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.202696+0000","last_change":"2024-09-11T05:12:35.202448+0000","last_active":"2024-09-11T05:12:35.202696+0000","last_peered":"2024-09-11T05:12:35.202696+0000","last_clean":"2024-09-11T05:12:35.202696+0000","last_became_active":"2024-09-11T05:12:35.200425+0000","last_became_peered":"2024-09-11T05:12:35.200425+0000","last_unstale":"2024-09-11T05:12:35.202696+0000","last_undegraded":"2024-09-11T05:12:35.202696+0000","last_fullsized":"2024-09-11T05:12:35.202696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:11.836522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197894+0000","last_change":"2024-09-11T05:12:31.170669+0000","last_active":"2024-09-11T05:12:35.197894+0000","last_peered":"2024-09-11T05:12:35.197894+0000","last_clean":"2024-09-11T05:12:35.197894+0000","last_became_active":"2024-09-11T05:12:31.170245+0000","last_became_peered":"2024-09-11T05:12:31.170245+0000","last_unstale":"2024-09-11T05:12:35.197894+0000","last_undegraded":"2024-09-11T05:12:35.197894+0000","last_fullsized":"2024-09-11T05:12:35.197894+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:41.806823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195910+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:35.195910+0000","last_peered":"2024-09-11T05:12:35.195910+0000","last_clean":"2024-09-11T05:12:35.195910+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:35.195910+0000","last_undegraded":"2024-09-11T05:12:35.195910+0000","last_fullsized":"2024-09-11T05:12:35.195910+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198180+0000","last_change":"2024-09-11T05:12:31.175435+0000","last_active":"2024-09-11T05:12:35.198180+0000","last_peered":"2024-09-11T05:12:35.198180+0000","last_clean":"2024-09-11T05:12:35.198180+0000","last_became_active":"2024-09-11T05:12:31.174826+0000","last_became_peered":"2024-09-11T05:12:31.174826+0000","last_unstale":"2024-09-11T05:12:35.198180+0000","last_undegraded":"2024-09-11T05:12:35.198180+0000","last_fullsized":"2024-09-11T05:12:35.198180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:26.974983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T05:12:34.190058+0000","last_change":"2024-09-11T05:12:33.176083+0000","last_active":"2024-09-11T05:12:18.438492+0000","last_peered":"2024-09-11T05:12:18.438492+0000","last_clean":"2024-09-11T05:12:18.438492+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T05:12:34.190058+0000","last_undegraded":"2024-09-11T05:12:34.190058+0000","last_fullsized":"2024-09-11T05:12:34.190058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197897+0000","last_change":"2024-09-11T05:12:31.173875+0000","last_active":"2024-09-11T05:12:35.197897+0000","last_peered":"2024-09-11T05:12:35.197897+0000","last_clean":"2024-09-11T05:12:35.197897+0000","last_became_active":"2024-09-11T05:12:31.173368+0000","last_became_peered":"2024-09-11T05:12:31.173368+0000","last_unstale":"2024-09-11T05:12:35.197897+0000","last_undegraded":"2024-09-11T05:12:35.197897+0000","last_fullsized":"2024-09-11T05:12:35.197897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:00.201066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198313+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:35.198313+0000","last_peered":"2024-09-11T05:12:35.198313+0000","last_clean":"2024-09-11T05:12:35.198313+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:35.198313+0000","last_undegraded":"2024-09-11T05:12:35.198313+0000","last_fullsized":"2024-09-11T05:12:35.198313+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209255+0000","last_change":"2024-09-11T05:12:35.208978+0000","last_active":"2024-09-11T05:12:35.209255+0000","last_peered":"2024-09-11T05:12:35.209255+0000","last_clean":"2024-09-11T05:12:35.209255+0000","last_became_active":"2024-09-11T05:12:35.208254+0000","last_became_peered":"2024-09-11T05:12:35.208254+0000","last_unstale":"2024-09-11T05:12:35.209255+0000","last_undegraded":"2024-09-11T05:12:35.209255+0000","last_fullsized":"2024-09-11T05:12:35.209255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:32.748794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"19'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195517+0000","last_change":"2024-09-11T05:12:31.174174+0000","last_active":"2024-09-11T05:12:35.195517+0000","last_peered":"2024-09-11T05:12:35.195517+0000","last_clean":"2024-09-11T05:12:35.195517+0000","last_became_active":"2024-09-11T05:12:31.173727+0000","last_became_peered":"2024-09-11T05:12:31.173727+0000","last_unstale":"2024-09-11T05:12:35.195517+0000","last_undegraded":"2024-09-11T05:12:35.195517+0000","last_fullsized":"2024-09-11T05:12:35.195517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:58.026864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":47516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":41407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:41.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:12:41.786 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:41.786 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:12:41.804 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-11T05:12:41.343944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54416,"kb_used_data":352,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689264,"statfs":{"total":193273528320,"available":193217806336,"internally_reserved":0,"allocated":360448,"data_stored":95032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001813"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.193745+0000","last_change":"2024-09-11T05:12:31.174556+0000","last_active":"2024-09-11T05:12:35.193745+0000","last_peered":"2024-09-11T05:12:35.193745+0000","last_clean":"2024-09-11T05:12:35.193745+0000","last_became_active":"2024-09-11T05:12:31.172693+0000","last_became_peered":"2024-09-11T05:12:31.172693+0000","last_unstale":"2024-09-11T05:12:35.193745+0000","last_undegraded":"2024-09-11T05:12:35.193745+0000","last_fullsized":"2024-09-11T05:12:35.193745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:12.712402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.205074+0000","last_change":"2024-09-11T05:12:35.204831+0000","last_active":"2024-09-11T05:12:35.205074+0000","last_peered":"2024-09-11T05:12:35.205074+0000","last_clean":"2024-09-11T05:12:35.205074+0000","last_became_active":"2024-09-11T05:12:35.203820+0000","last_became_peered":"2024-09-11T05:12:35.203820+0000","last_unstale":"2024-09-11T05:12:35.205074+0000","last_undegraded":"2024-09-11T05:12:35.205074+0000","last_fullsized":"2024-09-11T05:12:35.205074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:46.881983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209515+0000","last_change":"2024-09-11T05:12:35.208981+0000","last_active":"2024-09-11T05:12:35.209515+0000","last_peered":"2024-09-11T05:12:35.209515+0000","last_clean":"2024-09-11T05:12:35.209515+0000","last_became_active":"2024-09-11T05:12:35.208030+0000","last_became_peered":"2024-09-11T05:12:35.208030+0000","last_unstale":"2024-09-11T05:12:35.209515+0000","last_undegraded":"2024-09-11T05:12:35.209515+0000","last_fullsized":"2024-09-11T05:12:35.209515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:35.405376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.203271+0000","last_change":"2024-09-11T05:12:35.203015+0000","last_active":"2024-09-11T05:12:35.203271+0000","last_peered":"2024-09-11T05:12:35.203271+0000","last_clean":"2024-09-11T05:12:35.203271+0000","last_became_active":"2024-09-11T05:12:35.202004+0000","last_became_peered":"2024-09-11T05:12:35.202004+0000","last_unstale":"2024-09-11T05:12:35.203271+0000","last_undegraded":"2024-09-11T05:12:35.203271+0000","last_fullsized":"2024-09-11T05:12:35.203271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:02.610669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.204034+0000","last_change":"2024-09-11T05:12:35.203792+0000","last_active":"2024-09-11T05:12:35.204034+0000","last_peered":"2024-09-11T05:12:35.204034+0000","last_clean":"2024-09-11T05:12:35.204034+0000","last_became_active":"2024-09-11T05:12:35.203005+0000","last_became_peered":"2024-09-11T05:12:35.203005+0000","last_unstale":"2024-09-11T05:12:35.204034+0000","last_undegraded":"2024-09-11T05:12:35.204034+0000","last_fullsized":"2024-09-11T05:12:35.204034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:48.591394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195617+0000","last_change":"2024-09-11T05:12:31.176835+0000","last_active":"2024-09-11T05:12:35.195617+0000","last_peered":"2024-09-11T05:12:35.195617+0000","last_clean":"2024-09-11T05:12:35.195617+0000","last_became_active":"2024-09-11T05:12:31.176545+0000","last_became_peered":"2024-09-11T05:12:31.176545+0000","last_unstale":"2024-09-11T05:12:35.195617+0000","last_undegraded":"2024-09-11T05:12:35.195617+0000","last_fullsized":"2024-09-11T05:12:35.195617+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:45.064646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195994+0000","last_change":"2024-09-11T05:12:31.170829+0000","last_active":"2024-09-11T05:12:35.195994+0000","last_peered":"2024-09-11T05:12:35.195994+0000","last_clean":"2024-09-11T05:12:35.195994+0000","last_became_active":"2024-09-11T05:12:31.170224+0000","last_became_peered":"2024-09-11T05:12:31.170224+0000","last_unstale":"2024-09-11T05:12:35.195994+0000","last_undegraded":"2024-09-11T05:12:35.195994+0000","last_fullsized":"2024-09-11T05:12:35.195994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:42.377273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.205478+0000","last_change":"2024-09-11T05:12:35.205238+0000","last_active":"2024-09-11T05:12:35.205478+0000","last_peered":"2024-09-11T05:12:35.205478+0000","last_clean":"2024-09-11T05:12:35.205478+0000","last_became_active":"2024-09-11T05:12:35.201219+0000","last_became_peered":"2024-09-11T05:12:35.201219+0000","last_unstale":"2024-09-11T05:12:35.205478+0000","last_undegraded":"2024-09-11T05:12:35.205478+0000","last_fullsized":"2024-09-11T05:12:35.205478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:40:38.963154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.196633+0000","last_change":"2024-09-11T05:12:21.487931+0000","last_active":"2024-09-11T05:12:35.196633+0000","last_peered":"2024-09-11T05:12:35.196633+0000","last_clean":"2024-09-11T05:12:35.196633+0000","last_became_active":"2024-09-11T05:12:19.477152+0000","last_became_peered":"2024-09-11T05:12:19.477152+0000","last_unstale":"2024-09-11T05:12:35.196633+0000","last_undegraded":"2024-09-11T05:12:35.196633+0000","last_fullsized":"2024-09-11T05:12:35.196633+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:27.227389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209280+0000","last_change":"2024-09-11T05:12:35.209014+0000","last_active":"2024-09-11T05:12:35.209280+0000","last_peered":"2024-09-11T05:12:35.209280+0000","last_clean":"2024-09-11T05:12:35.209280+0000","last_became_active":"2024-09-11T05:12:35.208141+0000","last_became_peered":"2024-09-11T05:12:35.208141+0000","last_unstale":"2024-09-11T05:12:35.209280+0000","last_undegraded":"2024-09-11T05:12:35.209280+0000","last_fullsized":"2024-09-11T05:12:35.209280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:47.208445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.205358+0000","last_change":"2024-09-11T05:12:35.205215+0000","last_active":"2024-09-11T05:12:35.205358+0000","last_peered":"2024-09-11T05:12:35.205358+0000","last_clean":"2024-09-11T05:12:35.205358+0000","last_became_active":"2024-09-11T05:12:35.203248+0000","last_became_peered":"2024-09-11T05:12:35.203248+0000","last_unstale":"2024-09-11T05:12:35.205358+0000","last_undegraded":"2024-09-11T05:12:35.205358+0000","last_fullsized":"2024-09-11T05:12:35.205358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:52.778435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.191862+0000","last_change":"2024-09-11T05:12:31.177431+0000","last_active":"2024-09-11T05:12:35.191862+0000","last_peered":"2024-09-11T05:12:35.191862+0000","last_clean":"2024-09-11T05:12:35.191862+0000","last_became_active":"2024-09-11T05:12:31.175358+0000","last_became_peered":"2024-09-11T05:12:31.175358+0000","last_unstale":"2024-09-11T05:12:35.191862+0000","last_undegraded":"2024-09-11T05:12:35.191862+0000","last_fullsized":"2024-09-11T05:12:35.191862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:07.430680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":35,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.192104+0000","last_change":"2024-09-11T05:12:21.485209+0000","last_active":"2024-09-11T05:12:35.192104+0000","last_peered":"2024-09-11T05:12:35.192104+0000","last_clean":"2024-09-11T05:12:35.192104+0000","last_became_active":"2024-09-11T05:12:19.480133+0000","last_became_peered":"2024-09-11T05:12:19.480133+0000","last_unstale":"2024-09-11T05:12:35.192104+0000","last_undegraded":"2024-09-11T05:12:35.192104+0000","last_fullsized":"2024-09-11T05:12:35.192104+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:13.333964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.205228+0000","last_change":"2024-09-11T05:12:35.205045+0000","last_active":"2024-09-11T05:12:35.205228+0000","last_peered":"2024-09-11T05:12:35.205228+0000","last_clean":"2024-09-11T05:12:35.205228+0000","last_became_active":"2024-09-11T05:12:35.204315+0000","last_became_peered":"2024-09-11T05:12:35.204315+0000","last_unstale":"2024-09-11T05:12:35.205228+0000","last_undegraded":"2024-09-11T05:12:35.205228+0000","last_fullsized":"2024-09-11T05:12:35.205228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:01.399867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":44,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.216831+0000","last_change":"2024-09-11T05:12:35.216562+0000","last_active":"2024-09-11T05:12:35.216831+0000","last_peered":"2024-09-11T05:12:35.216831+0000","last_clean":"2024-09-11T05:12:35.216831+0000","last_became_active":"2024-09-11T05:12:35.211123+0000","last_became_peered":"2024-09-11T05:12:35.211123+0000","last_unstale":"2024-09-11T05:12:35.216831+0000","last_undegraded":"2024-09-11T05:12:35.216831+0000","last_fullsized":"2024-09-11T05:12:35.216831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:35.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.202696+0000","last_change":"2024-09-11T05:12:35.202448+0000","last_active":"2024-09-11T05:12:35.202696+0000","last_peered":"2024-09-11T05:12:35.202696+0000","last_clean":"2024-09-11T05:12:35.202696+0000","last_became_active":"2024-09-11T05:12:35.200425+0000","last_became_peered":"2024-09-11T05:12:35.200425+0000","last_unstale":"2024-09-11T05:12:35.202696+0000","last_undegraded":"2024-09-11T05:12:35.202696+0000","last_fullsized":"2024-09-11T05:12:35.202696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:11.836522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197894+0000","last_change":"2024-09-11T05:12:31.170669+0000","last_active":"2024-09-11T05:12:35.197894+0000","last_peered":"2024-09-11T05:12:35.197894+0000","last_clean":"2024-09-11T05:12:35.197894+0000","last_became_active":"2024-09-11T05:12:31.170245+0000","last_became_peered":"2024-09-11T05:12:31.170245+0000","last_unstale":"2024-09-11T05:12:35.197894+0000","last_undegraded":"2024-09-11T05:12:35.197894+0000","last_fullsized":"2024-09-11T05:12:35.197894+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:41.806823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195910+0000","last_change":"2024-09-11T05:12:21.489088+0000","last_active":"2024-09-11T05:12:35.195910+0000","last_peered":"2024-09-11T05:12:35.195910+0000","last_clean":"2024-09-11T05:12:35.195910+0000","last_became_active":"2024-09-11T05:12:19.477916+0000","last_became_peered":"2024-09-11T05:12:19.477916+0000","last_unstale":"2024-09-11T05:12:35.195910+0000","last_undegraded":"2024-09-11T05:12:35.195910+0000","last_fullsized":"2024-09-11T05:12:35.195910+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:57.146927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198180+0000","last_change":"2024-09-11T05:12:31.175435+0000","last_active":"2024-09-11T05:12:35.198180+0000","last_peered":"2024-09-11T05:12:35.198180+0000","last_clean":"2024-09-11T05:12:35.198180+0000","last_became_active":"2024-09-11T05:12:31.174826+0000","last_became_peered":"2024-09-11T05:12:31.174826+0000","last_unstale":"2024-09-11T05:12:35.198180+0000","last_undegraded":"2024-09-11T05:12:35.198180+0000","last_fullsized":"2024-09-11T05:12:35.198180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:26.974983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.204301+0000","last_change":"2024-09-11T05:12:35.204039+0000","last_active":"2024-09-11T05:12:35.204301+0000","last_peered":"2024-09-11T05:12:35.204301+0000","last_clean":"2024-09-11T05:12:35.204301+0000","last_became_active":"2024-09-11T05:12:35.203317+0000","last_became_peered":"2024-09-11T05:12:35.203317+0000","last_unstale":"2024-09-11T05:12:35.204301+0000","last_undegraded":"2024-09-11T05:12:35.204301+0000","last_fullsized":"2024-09-11T05:12:35.204301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:44.838371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.197897+0000","last_change":"2024-09-11T05:12:31.173875+0000","last_active":"2024-09-11T05:12:35.197897+0000","last_peered":"2024-09-11T05:12:35.197897+0000","last_clean":"2024-09-11T05:12:35.197897+0000","last_became_active":"2024-09-11T05:12:31.173368+0000","last_became_peered":"2024-09-11T05:12:31.173368+0000","last_unstale":"2024-09-11T05:12:35.197897+0000","last_undegraded":"2024-09-11T05:12:35.197897+0000","last_fullsized":"2024-09-11T05:12:35.197897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:00.201066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.198313+0000","last_change":"2024-09-11T05:12:21.488736+0000","last_active":"2024-09-11T05:12:35.198313+0000","last_peered":"2024-09-11T05:12:35.198313+0000","last_clean":"2024-09-11T05:12:35.198313+0000","last_became_active":"2024-09-11T05:12:19.477445+0000","last_became_peered":"2024-09-11T05:12:19.477445+0000","last_unstale":"2024-09-11T05:12:35.198313+0000","last_undegraded":"2024-09-11T05:12:35.198313+0000","last_fullsized":"2024-09-11T05:12:35.198313+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:04.431332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.209255+0000","last_change":"2024-09-11T05:12:35.208978+0000","last_active":"2024-09-11T05:12:35.209255+0000","last_peered":"2024-09-11T05:12:35.209255+0000","last_clean":"2024-09-11T05:12:35.209255+0000","last_became_active":"2024-09-11T05:12:35.208254+0000","last_became_peered":"2024-09-11T05:12:35.208254+0000","last_unstale":"2024-09-11T05:12:35.209255+0000","last_undegraded":"2024-09-11T05:12:35.209255+0000","last_fullsized":"2024-09-11T05:12:35.209255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:32.748794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"19'1","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T05:12:35.195517+0000","last_change":"2024-09-11T05:12:31.174174+0000","last_active":"2024-09-11T05:12:35.195517+0000","last_peered":"2024-09-11T05:12:35.195517+0000","last_clean":"2024-09-11T05:12:35.195517+0000","last_became_active":"2024-09-11T05:12:31.173727+0000","last_became_peered":"2024-09-11T05:12:31.173727+0000","last_unstale":"2024-09-11T05:12:35.195517+0000","last_undegraded":"2024-09-11T05:12:35.195517+0000","last_fullsized":"2024-09-11T05:12:35.195517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:58.026864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":47516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":47516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:12:41.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-11T05:12:41.805 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T05:12:41.805 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T05:12:41.850 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T05:12:41.850 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T05:12:41.934 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-09-11T05:12:41.935 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-09-11T05:12:42.019 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T05:12:42.020 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T05:12:42.100 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T05:12:42.100 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T05:12:42.186 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-09-11T05:12:42.186 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-09-11T05:12:42.268 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T05:12:42.268 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T05:12:42.357 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T05:12:42.358 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T05:12:42.440 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-09-11T05:12:42.440 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-09-11T05:12:42.518 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T05:12:42.518 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T05:12:42.605 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T05:12:42.606 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T05:12:42.643 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-09-11T05:12:42.644 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-09-11T05:12:42.726 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T05:12:42.726 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T05:12:42.812 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T05:12:42.812 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T05:12:42.897 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-09-11T05:12:42.897 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-09-11T05:12:42.978 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T05:12:42.979 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T05:12:43.061 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T05:12:43.062 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T05:12:43.154 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-09-11T05:12:43.155 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-09-11T05:12:43.234 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T05:12:43.235 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T05:12:43.317 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T05:12:43.317 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T05:12:43.402 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-09-11T05:12:43.402 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-09-11T05:12:43.483 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T05:12:43.483 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T05:12:43.564 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T05:12:43.564 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T05:12:43.649 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-09-11T05:12:43.650 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-09-11T05:12:43.731 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T05:12:43.732 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T05:12:43.817 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T05:12:43.817 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T05:12:43.902 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-09-11T05:12:43.902 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-09-11T05:12:43.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-11T05:12:44.176 INFO:teuthology.orchestra.run.smithi133.stdout:{} 2024-09-11T05:12:44.176 INFO:teuthology.orchestra.run.smithi133.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-11T05:12:44.192 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T05:12:44.192 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:12:44.247 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T05:12:44.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-11T05:12:44.552 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:12:44.548+0000 7f9b75702640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-11T05:12:44.552 INFO:teuthology.orchestra.run.smithi133.stderr:osd.0 is already down. 2024-09-11T05:12:44.569 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-11T05:12:44.569 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-11T05:12:45.464 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T05:12:45.464 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T05:12:45.512 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T05:12:45.512 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T05:12:45.552 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-11T05:12:45.552 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-11T05:12:45.633 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T05:12:45.633 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T05:12:45.718 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T05:12:45.718 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T05:12:45.797 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-11T05:12:45.797 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-11T05:12:45.886 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T05:12:45.887 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T05:12:45.974 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T05:12:45.975 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T05:12:46.057 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-11T05:12:46.057 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-11T05:12:46.138 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T05:12:46.138 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T05:12:46.222 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T05:12:46.222 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T05:12:46.301 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-11T05:12:46.302 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-11T05:12:46.382 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T05:12:46.383 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T05:12:46.466 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T05:12:46.466 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T05:12:46.546 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-11T05:12:46.546 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-11T05:12:46.626 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T05:12:46.626 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T05:12:46.705 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T05:12:46.706 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T05:12:46.784 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-11T05:12:46.784 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-11T05:12:46.864 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T05:12:46.864 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T05:12:46.945 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T05:12:46.946 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T05:12:47.024 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-11T05:12:47.024 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-11T05:12:47.107 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T05:12:47.107 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T05:12:47.184 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T05:12:47.185 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T05:12:47.269 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-11T05:12:47.270 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-11T05:12:47.359 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T05:12:47.359 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T05:12:47.439 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T05:12:47.439 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T05:12:47.517 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-09-11T05:12:47.517 INFO:tasks.rep_lost_unfound_delete:() 2024-09-11T05:12:47.517 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T05:12:47.518 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-11T05:12:47.562 INFO:tasks.ceph.osd.0:Started 2024-09-11T05:12:47.562 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T05:12:47.666 INFO:teuthology.orchestra.run.smithi133.stderr:no valid command found; 10 closest matches: 2024-09-11T05:12:47.666 INFO:teuthology.orchestra.run.smithi133.stderr:0 2024-09-11T05:12:47.666 INFO:teuthology.orchestra.run.smithi133.stderr:1 2024-09-11T05:12:47.666 INFO:teuthology.orchestra.run.smithi133.stderr:2 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:abort 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:assert 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:config diff 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:config diff get 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:config get 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:config help [] 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:config set ... 2024-09-11T05:12:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:admin_socket: invalid command 2024-09-11T05:12:47.669 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T05:12:47.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-11T05:12:48.618 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:12:48.616+0000 7faa18d8a7c0 -1 Falling back to public interface 2024-09-11T05:12:52.564 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:12:52.560+0000 7faa18d8a7c0 -1 osd.0 20 log_to_monitors true 2024-09-11T05:12:52.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T05:12:52.796 INFO:teuthology.orchestra.run.smithi133.stderr:no valid command found; 10 closest matches: 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:0 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:1 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:2 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:abort 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:assert 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:bluefs debug_inject_read_zeros 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:bluefs files list 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:bluefs stats 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:bluestore allocator dump block 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:bluestore allocator fragmentation block 2024-09-11T05:12:52.797 INFO:teuthology.orchestra.run.smithi133.stderr:admin_socket: invalid command 2024-09-11T05:12:52.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T05:12:52.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-11T05:12:53.431 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:12:53.428+0000 7faa10b6d640 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T05:12:57.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T05:12:57.933 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:12:57.933 INFO:teuthology.orchestra.run.smithi133.stdout: "ops": [], 2024-09-11T05:12:57.933 INFO:teuthology.orchestra.run.smithi133.stdout: "num_ops": 0 2024-09-11T05:12:57.933 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:12:57.947 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-11T05:12:58.256 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:12:58.252+0000 7f9b75702640 -1 mon.a@0(leader).osd e25 definitely_dead 0 2024-09-11T05:12:58.256 INFO:teuthology.orchestra.run.smithi133.stderr:osd.0 is already in. 2024-09-11T05:12:58.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-09-11T05:12:58.273 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:12:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:12:58.561 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":25,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:12:55.419887+0000","last_up_change":"2024-09-11T05:12:54.417081+0000","last_in_change":"2024-09-11T05:12:34.169975+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T05:12:17.695239+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T05:12:29.395526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":24,"up_thru":24,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":3512043189}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":3512043189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":3512043189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":3512043189}]},"public_addr":"172.21.15.133:6804/3512043189","cluster_addr":"172.21.15.133:6805/3512043189","heartbeat_back_addr":"172.21.15.133:6807/3512043189","heartbeat_front_addr":"172.21.15.133:6806/3512043189","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":3149929203}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":3149929203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":3149929203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":3149929203}]},"public_addr":"172.21.15.133:6800/3149929203","cluster_addr":"172.21.15.133:6801/3149929203","heartbeat_back_addr":"172.21.15.133:6803/3149929203","heartbeat_front_addr":"172.21.15.133:6802/3149929203","state":["exists","up"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":841703869}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":841703869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":841703869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":841703869}]},"public_addr":"172.21.15.133:6804/841703869","cluster_addr":"172.21.15.133:6805/841703869","heartbeat_back_addr":"172.21.15.133:6807/841703869","heartbeat_front_addr":"172.21.15.133:6806/841703869","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":18}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:12:58.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-09-11T05:12:58.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T05:12:58.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T05:12:58.732 INFO:teuthology.orchestra.run.smithi133.stdout:103079215107 2024-09-11T05:12:58.733 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:12:58.741 INFO:teuthology.orchestra.run.smithi133.stdout:34359738383 2024-09-11T05:12:58.741 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:12:59.031 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-09-11T05:12:59.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215107 got 1 for osd.0 2024-09-11T05:12:59.081 INFO:teuthology.orchestra.run.smithi133.stdout:34359738382 2024-09-11T05:12:59.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738383 got 34359738382 for osd.1 2024-09-11T05:13:00.049 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:13:00.098 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T05:13:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:103079215107 2024-09-11T05:13:00.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215107 got 103079215107 for osd.0 2024-09-11T05:13:00.367 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:34359738383 2024-09-11T05:13:00.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738383 got 34359738383 for osd.1 2024-09-11T05:13:00.415 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:00.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-11T05:13:00.416 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:00.735 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:00.735 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:00.751 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-11T05:12:59.349503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55096,"kb_used_data":576,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688584,"statfs":{"total":193273528320,"available":193217110016,"internally_reserved":0,"allocated":589824,"data_stored":148212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6032,"internal_metadata":55765104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002330"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.455516+0000","last_change":"2024-09-11T05:12:55.455309+0000","last_active":"2024-09-11T05:12:55.455516+0000","last_peered":"2024-09-11T05:12:55.455516+0000","last_clean":"2024-09-11T05:12:55.455516+0000","last_became_active":"2024-09-11T05:12:55.454598+0000","last_became_peered":"2024-09-11T05:12:55.454598+0000","last_unstale":"2024-09-11T05:12:55.455516+0000","last_undegraded":"2024-09-11T05:12:55.455516+0000","last_fullsized":"2024-09-11T05:12:55.455516+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:18.608073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'4","reported_seq":24,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.455055+0000","last_change":"2024-09-11T05:12:55.455567+0000","last_active":"2024-09-11T05:12:55.455055+0000","last_peered":"2024-09-11T05:12:55.455055+0000","last_clean":"2024-09-11T05:12:42.606590+0000","last_became_active":"2024-09-11T05:12:55.454822+0000","last_became_peered":"2024-09-11T05:12:55.454822+0000","last_unstale":"2024-09-11T05:12:55.455055+0000","last_undegraded":"2024-09-11T05:12:55.434432+0000","last_fullsized":"2024-09-11T05:12:55.455055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":44,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.452144+0000","last_change":"2024-09-11T05:12:55.453563+0000","last_active":"2024-09-11T05:12:55.452144+0000","last_peered":"2024-09-11T05:12:55.452144+0000","last_clean":"2024-09-11T05:12:43.483894+0000","last_became_active":"2024-09-11T05:12:55.451925+0000","last_became_peered":"2024-09-11T05:12:55.451925+0000","last_unstale":"2024-09-11T05:12:55.452144+0000","last_undegraded":"2024-09-11T05:12:55.436234+0000","last_fullsized":"2024-09-11T05:12:55.452144+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.448771+0000","last_change":"2024-09-11T05:12:55.448487+0000","last_active":"2024-09-11T05:12:55.448771+0000","last_peered":"2024-09-11T05:12:55.448771+0000","last_clean":"2024-09-11T05:12:55.448771+0000","last_became_active":"2024-09-11T05:12:55.447728+0000","last_became_peered":"2024-09-11T05:12:55.447728+0000","last_unstale":"2024-09-11T05:12:55.448771+0000","last_undegraded":"2024-09-11T05:12:55.448771+0000","last_fullsized":"2024-09-11T05:12:55.448771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:55:25.701451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":35,"reported_epoch":25,"state":"active+recovering+degraded","last_fresh":"2024-09-11T05:12:55.446964+0000","last_change":"2024-09-11T05:12:55.456197+0000","last_active":"2024-09-11T05:12:55.446964+0000","last_peered":"2024-09-11T05:12:55.446964+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:12:55.446389+0000","last_became_peered":"2024-09-11T05:12:55.446389+0000","last_unstale":"2024-09-11T05:12:55.446964+0000","last_undegraded":"2024-09-11T05:12:55.432148+0000","last_fullsized":"2024-09-11T05:12:55.446964+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"22'3","reported_seq":39,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.451426+0000","last_change":"2024-09-11T05:12:55.452211+0000","last_active":"2024-09-11T05:12:55.451426+0000","last_peered":"2024-09-11T05:12:55.451426+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:12:55.451155+0000","last_became_peered":"2024-09-11T05:12:55.451155+0000","last_unstale":"2024-09-11T05:12:55.451426+0000","last_undegraded":"2024-09-11T05:12:55.433871+0000","last_fullsized":"2024-09-11T05:12:55.451426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"22'4","reported_seq":40,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.458278+0000","last_change":"2024-09-11T05:12:55.459076+0000","last_active":"2024-09-11T05:12:55.458278+0000","last_peered":"2024-09-11T05:12:55.458278+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:12:55.458073+0000","last_became_peered":"2024-09-11T05:12:55.458073+0000","last_unstale":"2024-09-11T05:12:55.458278+0000","last_undegraded":"2024-09-11T05:12:55.438245+0000","last_fullsized":"2024-09-11T05:12:55.458278+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"22'1","reported_seq":31,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.446755+0000","last_change":"2024-09-11T05:12:55.447233+0000","last_active":"2024-09-11T05:12:55.446755+0000","last_peered":"2024-09-11T05:12:55.446755+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:12:55.446511+0000","last_became_peered":"2024-09-11T05:12:55.446511+0000","last_unstale":"2024-09-11T05:12:55.446755+0000","last_undegraded":"2024-09-11T05:12:55.432221+0000","last_fullsized":"2024-09-11T05:12:55.446755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.453751+0000","last_change":"2024-09-11T05:12:55.452999+0000","last_active":"2024-09-11T05:12:55.453751+0000","last_peered":"2024-09-11T05:12:55.453751+0000","last_clean":"2024-09-11T05:12:55.453751+0000","last_became_active":"2024-09-11T05:12:55.452074+0000","last_became_peered":"2024-09-11T05:12:55.452074+0000","last_unstale":"2024-09-11T05:12:55.453751+0000","last_undegraded":"2024-09-11T05:12:55.453751+0000","last_fullsized":"2024-09-11T05:12:55.453751+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:12.789262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"22'5","reported_seq":46,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.457909+0000","last_change":"2024-09-11T05:12:55.458302+0000","last_active":"2024-09-11T05:12:55.457909+0000","last_peered":"2024-09-11T05:12:55.457909+0000","last_clean":"2024-09-11T05:12:43.565164+0000","last_became_active":"2024-09-11T05:12:55.457684+0000","last_became_peered":"2024-09-11T05:12:55.457684+0000","last_unstale":"2024-09-11T05:12:55.457909+0000","last_undegraded":"2024-09-11T05:12:55.439589+0000","last_fullsized":"2024-09-11T05:12:55.457909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.448658+0000","last_change":"2024-09-11T05:12:55.448399+0000","last_active":"2024-09-11T05:12:55.448658+0000","last_peered":"2024-09-11T05:12:55.448658+0000","last_clean":"2024-09-11T05:12:55.448658+0000","last_became_active":"2024-09-11T05:12:55.447336+0000","last_became_peered":"2024-09-11T05:12:55.447336+0000","last_unstale":"2024-09-11T05:12:55.448658+0000","last_undegraded":"2024-09-11T05:12:55.448658+0000","last_fullsized":"2024-09-11T05:12:55.448658+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:56.221032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'6","reported_seq":31,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.447843+0000","last_change":"2024-09-11T05:12:55.448292+0000","last_active":"2024-09-11T05:12:55.447843+0000","last_peered":"2024-09-11T05:12:55.447843+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:12:55.447588+0000","last_became_peered":"2024-09-11T05:12:55.447588+0000","last_unstale":"2024-09-11T05:12:55.447843+0000","last_undegraded":"2024-09-11T05:12:55.433063+0000","last_fullsized":"2024-09-11T05:12:55.447843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":38,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.447366+0000","last_change":"2024-09-11T05:12:55.447081+0000","last_active":"2024-09-11T05:12:55.447366+0000","last_peered":"2024-09-11T05:12:55.447366+0000","last_clean":"2024-09-11T05:12:55.447366+0000","last_became_active":"2024-09-11T05:12:55.446404+0000","last_became_peered":"2024-09-11T05:12:55.446404+0000","last_unstale":"2024-09-11T05:12:55.447366+0000","last_undegraded":"2024-09-11T05:12:55.447366+0000","last_fullsized":"2024-09-11T05:12:55.447366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:08.117948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'3","reported_seq":34,"reported_epoch":25,"state":"active+recovering+degraded","last_fresh":"2024-09-11T05:12:55.452182+0000","last_change":"2024-09-11T05:12:55.461772+0000","last_active":"2024-09-11T05:12:55.452182+0000","last_peered":"2024-09-11T05:12:55.452182+0000","last_clean":"2024-09-11T05:12:42.358674+0000","last_became_active":"2024-09-11T05:12:55.451416+0000","last_became_peered":"2024-09-11T05:12:55.451416+0000","last_unstale":"2024-09-11T05:12:55.452182+0000","last_undegraded":"2024-09-11T05:12:55.433888+0000","last_fullsized":"2024-09-11T05:12:55.452182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":63,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.452938+0000","last_change":"2024-09-11T05:12:55.452649+0000","last_active":"2024-09-11T05:12:55.452938+0000","last_peered":"2024-09-11T05:12:55.452938+0000","last_clean":"2024-09-11T05:12:55.452938+0000","last_became_active":"2024-09-11T05:12:55.451742+0000","last_became_peered":"2024-09-11T05:12:55.451742+0000","last_unstale":"2024-09-11T05:12:55.452938+0000","last_undegraded":"2024-09-11T05:12:55.452938+0000","last_fullsized":"2024-09-11T05:12:55.452938+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:33.633362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":34,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.452850+0000","last_change":"2024-09-11T05:12:55.453341+0000","last_active":"2024-09-11T05:12:55.452850+0000","last_peered":"2024-09-11T05:12:55.452850+0000","last_clean":"2024-09-11T05:12:42.979390+0000","last_became_active":"2024-09-11T05:12:55.452624+0000","last_became_peered":"2024-09-11T05:12:55.452624+0000","last_unstale":"2024-09-11T05:12:55.452850+0000","last_undegraded":"2024-09-11T05:12:55.434578+0000","last_fullsized":"2024-09-11T05:12:55.452850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"22'1","reported_seq":36,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.456823+0000","last_change":"2024-09-11T05:12:55.457367+0000","last_active":"2024-09-11T05:12:55.456823+0000","last_peered":"2024-09-11T05:12:55.456823+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:12:55.456596+0000","last_became_peered":"2024-09-11T05:12:55.456596+0000","last_unstale":"2024-09-11T05:12:55.456823+0000","last_undegraded":"2024-09-11T05:12:55.438590+0000","last_fullsized":"2024-09-11T05:12:55.456823+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.456851+0000","last_change":"2024-09-11T05:12:55.456617+0000","last_active":"2024-09-11T05:12:55.456851+0000","last_peered":"2024-09-11T05:12:55.456851+0000","last_clean":"2024-09-11T05:12:55.456851+0000","last_became_active":"2024-09-11T05:12:55.455587+0000","last_became_peered":"2024-09-11T05:12:55.455587+0000","last_unstale":"2024-09-11T05:12:55.456851+0000","last_undegraded":"2024-09-11T05:12:55.456851+0000","last_fullsized":"2024-09-11T05:12:55.456851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:01.760829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'5","reported_seq":41,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.452859+0000","last_change":"2024-09-11T05:12:55.453263+0000","last_active":"2024-09-11T05:12:55.452859+0000","last_peered":"2024-09-11T05:12:55.452859+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:12:55.452650+0000","last_became_peered":"2024-09-11T05:12:55.452650+0000","last_unstale":"2024-09-11T05:12:55.452859+0000","last_undegraded":"2024-09-11T05:12:55.436532+0000","last_fullsized":"2024-09-11T05:12:55.452859+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.448925+0000","last_change":"2024-09-11T05:12:55.448699+0000","last_active":"2024-09-11T05:12:55.448925+0000","last_peered":"2024-09-11T05:12:55.448925+0000","last_clean":"2024-09-11T05:12:55.448925+0000","last_became_active":"2024-09-11T05:12:55.447912+0000","last_became_peered":"2024-09-11T05:12:55.447912+0000","last_unstale":"2024-09-11T05:12:55.448925+0000","last_undegraded":"2024-09-11T05:12:55.448925+0000","last_fullsized":"2024-09-11T05:12:55.448925+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:25.741128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"22'2","reported_seq":37,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.453161+0000","last_change":"2024-09-11T05:12:55.453710+0000","last_active":"2024-09-11T05:12:55.453161+0000","last_peered":"2024-09-11T05:12:55.453161+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:12:55.452955+0000","last_became_peered":"2024-09-11T05:12:55.452955+0000","last_unstale":"2024-09-11T05:12:55.453161+0000","last_undegraded":"2024-09-11T05:12:55.437265+0000","last_fullsized":"2024-09-11T05:12:55.453161+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.455489+0000","last_change":"2024-09-11T05:12:55.455250+0000","last_active":"2024-09-11T05:12:55.455489+0000","last_peered":"2024-09-11T05:12:55.455489+0000","last_clean":"2024-09-11T05:12:55.455489+0000","last_became_active":"2024-09-11T05:12:55.454523+0000","last_became_peered":"2024-09-11T05:12:55.454523+0000","last_unstale":"2024-09-11T05:12:55.455489+0000","last_undegraded":"2024-09-11T05:12:55.455489+0000","last_fullsized":"2024-09-11T05:12:55.455489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:01:10.579372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-11T05:12:55.457266+0000","last_change":"2024-09-11T05:12:55.457029+0000","last_active":"2024-09-11T05:12:55.457266+0000","last_peered":"2024-09-11T05:12:55.457266+0000","last_clean":"2024-09-11T05:12:55.457266+0000","last_became_active":"2024-09-11T05:12:55.456346+0000","last_became_peered":"2024-09-11T05:12:55.456346+0000","last_unstale":"2024-09-11T05:12:55.457266+0000","last_undegraded":"2024-09-11T05:12:55.457266+0000","last_fullsized":"2024-09-11T05:12:55.457266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:23.237509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"22'9","reported_seq":45,"reported_epoch":25,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:12:55.454834+0000","last_change":"2024-09-11T05:12:55.456185+0000","last_active":"2024-09-11T05:12:55.454834+0000","last_peered":"2024-09-11T05:12:55.454834+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:12:55.454607+0000","last_became_peered":"2024-09-11T05:12:55.454607+0000","last_unstale":"2024-09-11T05:12:55.454834+0000","last_undegraded":"2024-09-11T05:12:55.437056+0000","last_fullsized":"2024-09-11T05:12:55.454834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738383,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":331776,"data_stored":79659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215107,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":252,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":258048,"data_stored":68553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:00.753 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:13:01.100 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:01.100 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":58,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":25,"num_osds":3,"num_up_osds":2,"osd_up_since":1726031574,"num_in_osds":2,"osd_in_since":1726031554,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56418304,"bytes_avail":193217110016,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-11T05:12:02.237066+0000","services":{}},"progress_events":{}} 2024-09-11T05:13:01.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '40251081-3433-4f55-b0e4-ce0789fd19fd', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 58, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 25, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726031574, 'num_in_osds': 2, 'osd_in_since': 1726031554, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56418304, 'bytes_avail': 193217110016, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T05:12:02:199900+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-11T05:12:02.237066+0000', 'services': {}}, 'progress_events': {}} 2024-09-11T05:13:01.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-11T05:13:01.117 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T05:13:01.117 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:13:07.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-11T05:13:07.219 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T05:13:07.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-11T05:13:07.536 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:13:07.536+0000 7f9b75702640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-11T05:13:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:osd.1 is already down. 2024-09-11T05:13:07.554 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-11T05:13:07.856 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:13:07.856+0000 7f9b75702640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-11T05:13:08.478 INFO:teuthology.orchestra.run.smithi133.stderr:marked out osd.1. 2024-09-11T05:13:08.503 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-11T05:13:09.499 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.496+0000 7fa9fb331640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T05:13:09.499 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.496+0000 7fa9ff339640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T05:13:09.499 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.496+0000 7fa9fab30640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T05:13:09.501 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.496+0000 7fa9fab30640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T05:13:09.501 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9fa32f640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T05:13:09.502 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9f9b2e640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T05:13:09.502 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9f832b640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T05:13:09.502 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9fcb34640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T05:13:09.502 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9f9b2e640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T05:13:09.502 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9fa32f640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T05:13:09.502 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9fcb34640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T05:13:09.502 INFO:teuthology.orchestra.run.smithi133.stderr:marked osd lost in epoch 26 2024-09-11T05:13:09.503 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9fd335640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T05:13:09.503 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9f832b640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T05:13:09.503 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:09.500+0000 7fa9f7b2a640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T05:13:09.524 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T05:13:09.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T05:13:09.526 INFO:tasks.ceph.osd.2:Started 2024-09-11T05:13:09.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T05:13:09.641 INFO:teuthology.orchestra.run.smithi133.stderr:no valid command found; 10 closest matches: 2024-09-11T05:13:09.641 INFO:teuthology.orchestra.run.smithi133.stderr:0 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:1 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:2 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:abort 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:assert 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:config diff 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:config diff get 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:config get 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:config help [] 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:config set ... 2024-09-11T05:13:09.642 INFO:teuthology.orchestra.run.smithi133.stderr:admin_socket: invalid command 2024-09-11T05:13:09.644 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T05:13:09.644 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-11T05:13:10.594 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-09-11T05:13:10.592+0000 7f4ac68087c0 -1 Falling back to public interface 2024-09-11T05:13:12.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-11T05:13:14.467 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-09-11T05:13:14.468+0000 7f4ac68087c0 -1 osd.2 17 log_to_monitors true 2024-09-11T05:13:14.646 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T05:13:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:no valid command found; 10 closest matches: 2024-09-11T05:13:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:0 2024-09-11T05:13:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:1 2024-09-11T05:13:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:2 2024-09-11T05:13:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:abort 2024-09-11T05:13:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:assert 2024-09-11T05:13:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:bluefs debug_inject_read_zeros 2024-09-11T05:13:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:bluefs files list 2024-09-11T05:13:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:bluefs stats 2024-09-11T05:13:14.772 INFO:teuthology.orchestra.run.smithi133.stderr:bluestore allocator dump block 2024-09-11T05:13:14.773 INFO:teuthology.orchestra.run.smithi133.stderr:bluestore allocator fragmentation block 2024-09-11T05:13:14.773 INFO:teuthology.orchestra.run.smithi133.stderr:admin_socket: invalid command 2024-09-11T05:13:14.774 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T05:13:14.775 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-11T05:13:15.421 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9ff339640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T05:13:15.422 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9fb331640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T05:13:15.422 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9feb38640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T05:13:15.423 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9fab30640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T05:13:15.423 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9fe337640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T05:13:15.423 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9fe337640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T05:13:15.424 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9fdb36640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T05:13:15.424 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9f9b2e640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T05:13:15.424 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.423+0000 7fa9fd335640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T05:13:15.425 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.427+0000 7fa9fcb34640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T05:13:15.426 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.427+0000 7fa9f832b640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T05:13:15.426 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.427+0000 7fa9fcb34640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T05:13:15.426 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.427+0000 7fa9f832b640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T05:13:15.426 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:15.427+0000 7fa9f7b2a640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T05:13:16.432 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-09-11T05:13:16.431+0000 7f4aba5e3640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T05:13:17.463 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9ff339640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T05:13:17.463 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9f9b2e640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T05:13:17.463 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9feb38640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T05:13:17.463 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9fcb34640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T05:13:17.464 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9fd335640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T05:13:17.464 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9f832b640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T05:13:17.464 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9fe337640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T05:13:17.465 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9fdb36640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T05:13:17.465 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.463+0000 7fa9fb331640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T05:13:17.466 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.467+0000 7fa9f8b2c640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T05:13:17.466 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.467+0000 7fa9feb38640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T05:13:17.467 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.467+0000 7fa9f7b2a640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T05:13:17.467 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.467+0000 7fa9f832b640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T05:13:17.467 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:17.467+0000 7fa9fe337640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T05:13:18.484 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:18.483+0000 7fa9fab30640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T05:13:18.485 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:18.483+0000 7fa9fe337640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T05:13:18.485 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:18.483+0000 7fa9fd335640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T05:13:18.485 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:18.483+0000 7fa9fa32f640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T05:13:18.487 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:18.487+0000 7fa9fc333640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T05:13:18.487 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:18.487+0000 7fa9fbb32640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T05:13:18.488 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:18.487+0000 7fa9f8b2c640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T05:13:19.488 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:19.487+0000 7fa9fab30640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T05:13:19.489 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:19.487+0000 7fa9fa32f640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T05:13:19.490 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:19.487+0000 7fa9fe337640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T05:13:19.490 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:19.487+0000 7fa9f8b2c640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T05:13:19.490 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:19.487+0000 7fa9fd335640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T05:13:19.491 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:19.487+0000 7fa9fc333640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T05:13:19.491 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:19.487+0000 7fa9f7b2a640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T05:13:19.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T05:13:19.906 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:19.907 INFO:teuthology.orchestra.run.smithi133.stdout: "ops": [], 2024-09-11T05:13:19.907 INFO:teuthology.orchestra.run.smithi133.stdout: "num_ops": 0 2024-09-11T05:13:19.907 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:19.920 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-11T05:13:20.232 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:13:20.231+0000 7f9b75702640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2024-09-11T05:13:20.478 INFO:teuthology.orchestra.run.smithi133.stderr:marked in osd.2. 2024-09-11T05:13:20.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-09-11T05:13:20.508 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:13:20.804 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:20.805 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":34,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:13:20.478012+0000","last_up_change":"2024-09-11T05:13:17.437886+0000","last_in_change":"2024-09-11T05:13:20.478012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T05:12:17.695239+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T05:12:29.395526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":24,"up_thru":32,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":3512043189}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":3512043189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":3512043189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":3512043189}]},"public_addr":"172.21.15.133:6804/3512043189","cluster_addr":"172.21.15.133:6805/3512043189","heartbeat_back_addr":"172.21.15.133:6807/3512043189","heartbeat_front_addr":"172.21.15.133:6806/3512043189","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":24,"down_at":26,"lost_at":26,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":3149929203}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":3149929203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":3149929203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":3149929203}]},"public_addr":"172.21.15.133:6800/3149929203","cluster_addr":"172.21.15.133:6801/3149929203","heartbeat_back_addr":"172.21.15.133:6803/3149929203","heartbeat_front_addr":"172.21.15.133:6802/3149929203","state":["exists"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":15,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":401696059}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":401696059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":401696059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":401696059}]},"public_addr":"172.21.15.133:6800/401696059","cluster_addr":"172.21.15.133:6801/401696059","heartbeat_back_addr":"172.21.15.133:6803/401696059","heartbeat_front_addr":"172.21.15.133:6802/401696059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":26},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":18}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:13:20.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-09-11T05:13:20.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T05:13:20.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T05:13:20.982 INFO:teuthology.orchestra.run.smithi133.stdout:133143986179 2024-09-11T05:13:20.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:13:20.983 INFO:teuthology.orchestra.run.smithi133.stdout:103079215112 2024-09-11T05:13:20.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:13:21.319 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-09-11T05:13:21.326 INFO:teuthology.orchestra.run.smithi133.stdout:103079215111 2024-09-11T05:13:21.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 1 for osd.2 2024-09-11T05:13:21.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215112 got 103079215111 for osd.0 2024-09-11T05:13:22.338 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:13:22.345 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:13:22.491 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9fb331640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T05:13:22.491 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9fa32f640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T05:13:22.491 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9feb38640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T05:13:22.493 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9fe337640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T05:13:22.493 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9f9b2e640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T05:13:22.493 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9f932d640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T05:13:22.493 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9f8b2c640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T05:13:22.493 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9f832b640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T05:13:22.493 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-11T05:13:22.491+0000 7fa9f7b2a640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T05:13:22.687 INFO:teuthology.orchestra.run.smithi133.stdout:133143986179 2024-09-11T05:13:22.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 133143986179 for osd.2 2024-09-11T05:13:22.704 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:22.709 INFO:teuthology.orchestra.run.smithi133.stdout:103079215112 2024-09-11T05:13:22.726 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215112 got 103079215112 for osd.0 2024-09-11T05:13:22.727 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:22.727 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-11T05:13:22.727 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:23.059 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:23.059 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:23.076 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-11T05:13:22.487003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55616,"kb_used_data":588,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688064,"statfs":{"total":193273528320,"available":193216577536,"internally_reserved":0,"allocated":602112,"data_stored":179303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":56286550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015792"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.490604+0000","last_change":"2024-09-11T05:13:20.499124+0000","last_active":"2024-09-11T05:13:20.499124+0000","last_peered":"2024-09-11T05:13:19.490604+0000","last_clean":"2024-09-11T05:12:55.457650+0000","last_became_active":"2024-09-11T05:13:02.466134+0000","last_became_peered":"2024-09-11T05:13:02.466134+0000","last_unstale":"2024-09-11T05:13:19.490604+0000","last_undegraded":"2024-09-11T05:13:19.490604+0000","last_fullsized":"2024-09-11T05:13:02.460518+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'4","reported_seq":46,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.495347+0000","last_change":"2024-09-11T05:13:20.498602+0000","last_active":"2024-09-11T05:13:20.498602+0000","last_peered":"2024-09-11T05:13:19.495347+0000","last_clean":"2024-09-11T05:12:42.606590+0000","last_became_active":"2024-09-11T05:13:19.494550+0000","last_became_peered":"2024-09-11T05:13:19.494550+0000","last_unstale":"2024-09-11T05:13:19.495347+0000","last_undegraded":"2024-09-11T05:13:19.486154+0000","last_fullsized":"2024-09-11T05:13:19.495347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":62,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.500074+0000","last_change":"2024-09-11T05:13:20.491369+0000","last_active":"2024-09-11T05:13:20.491369+0000","last_peered":"2024-09-11T05:13:19.500074+0000","last_clean":"2024-09-11T05:12:43.483894+0000","last_became_active":"2024-09-11T05:13:19.499437+0000","last_became_peered":"2024-09-11T05:13:19.499437+0000","last_unstale":"2024-09-11T05:13:19.500074+0000","last_undegraded":"2024-09-11T05:13:19.487419+0000","last_fullsized":"2024-09-11T05:13:19.500074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.496251+0000","last_change":"2024-09-11T05:13:20.497118+0000","last_active":"2024-09-11T05:13:20.497118+0000","last_peered":"2024-09-11T05:13:19.496251+0000","last_clean":"2024-09-11T05:13:19.496251+0000","last_became_active":"2024-09-11T05:13:19.493075+0000","last_became_peered":"2024-09-11T05:13:19.493075+0000","last_unstale":"2024-09-11T05:13:19.496251+0000","last_undegraded":"2024-09-11T05:13:19.496251+0000","last_fullsized":"2024-09-11T05:13:19.496251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":58,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-11T05:13:20.488921+0000","last_change":"2024-09-11T05:13:20.488921+0000","last_active":"2024-09-11T05:13:20.488921+0000","last_peered":"2024-09-11T05:13:19.496722+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:13:19.495650+0000","last_became_peered":"2024-09-11T05:13:19.495650+0000","last_unstale":"2024-09-11T05:13:20.488921+0000","last_undegraded":"2024-09-11T05:13:20.488921+0000","last_fullsized":"2024-09-11T05:13:20.488921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"22'3","reported_seq":54,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.488895+0000","last_change":"2024-09-11T05:13:20.490496+0000","last_active":"2024-09-11T05:13:20.490496+0000","last_peered":"2024-09-11T05:13:19.488895+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:13:02.465190+0000","last_became_peered":"2024-09-11T05:13:02.465190+0000","last_unstale":"2024-09-11T05:13:19.488895+0000","last_undegraded":"2024-09-11T05:13:02.459555+0000","last_fullsized":"2024-09-11T05:13:02.459067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"22'4","reported_seq":55,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.488609+0000","last_change":"2024-09-11T05:13:20.491080+0000","last_active":"2024-09-11T05:13:20.491080+0000","last_peered":"2024-09-11T05:13:19.488609+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:13:02.463860+0000","last_became_peered":"2024-09-11T05:13:02.463860+0000","last_unstale":"2024-09-11T05:13:19.488609+0000","last_undegraded":"2024-09-11T05:13:02.458207+0000","last_fullsized":"2024-09-11T05:13:02.457873+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'1","reported_seq":55,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-11T05:13:20.494249+0000","last_change":"2024-09-11T05:13:20.494249+0000","last_active":"2024-09-11T05:13:20.494249+0000","last_peered":"2024-09-11T05:13:19.496706+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:13:19.496020+0000","last_became_peered":"2024-09-11T05:13:19.496020+0000","last_unstale":"2024-09-11T05:13:20.494249+0000","last_undegraded":"2024-09-11T05:13:20.494249+0000","last_fullsized":"2024-09-11T05:13:20.494249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.488080+0000","last_change":"2024-09-11T05:13:20.490430+0000","last_active":"2024-09-11T05:13:20.490430+0000","last_peered":"2024-09-11T05:13:19.488080+0000","last_clean":"2024-09-11T05:12:35.196749+0000","last_became_active":"2024-09-11T05:13:02.460908+0000","last_became_peered":"2024-09-11T05:13:02.460908+0000","last_unstale":"2024-09-11T05:13:19.488080+0000","last_undegraded":"2024-09-11T05:13:19.488080+0000","last_fullsized":"2024-09-11T05:13:02.456864+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"22'5","reported_seq":64,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.500015+0000","last_change":"2024-09-11T05:13:20.489110+0000","last_active":"2024-09-11T05:13:20.489110+0000","last_peered":"2024-09-11T05:13:19.500015+0000","last_clean":"2024-09-11T05:12:43.565164+0000","last_became_active":"2024-09-11T05:13:19.499294+0000","last_became_peered":"2024-09-11T05:13:19.499294+0000","last_unstale":"2024-09-11T05:13:19.500015+0000","last_undegraded":"2024-09-11T05:13:19.487494+0000","last_fullsized":"2024-09-11T05:13:19.500015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.499766+0000","last_change":"2024-09-11T05:13:20.489707+0000","last_active":"2024-09-11T05:13:20.489707+0000","last_peered":"2024-09-11T05:13:19.499766+0000","last_clean":"2024-09-11T05:13:19.499766+0000","last_became_active":"2024-09-11T05:13:19.495911+0000","last_became_peered":"2024-09-11T05:13:19.495911+0000","last_unstale":"2024-09-11T05:13:19.499766+0000","last_undegraded":"2024-09-11T05:13:19.499766+0000","last_fullsized":"2024-09-11T05:13:19.499766+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'6","reported_seq":50,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.489125+0000","last_change":"2024-09-11T05:13:20.498915+0000","last_active":"2024-09-11T05:13:20.498915+0000","last_peered":"2024-09-11T05:13:19.489125+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:13:02.467742+0000","last_became_peered":"2024-09-11T05:13:02.467742+0000","last_unstale":"2024-09-11T05:13:19.489125+0000","last_undegraded":"2024-09-11T05:13:02.461565+0000","last_fullsized":"2024-09-11T05:13:02.461328+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":58,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.488433+0000","last_change":"2024-09-11T05:13:20.490540+0000","last_active":"2024-09-11T05:13:20.490540+0000","last_peered":"2024-09-11T05:13:19.488433+0000","last_clean":"2024-09-11T05:12:55.450586+0000","last_became_active":"2024-09-11T05:13:02.462780+0000","last_became_peered":"2024-09-11T05:13:02.462780+0000","last_unstale":"2024-09-11T05:13:19.488433+0000","last_undegraded":"2024-09-11T05:13:02.457974+0000","last_fullsized":"2024-09-11T05:13:02.457836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'3","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.500708+0000","last_change":"2024-09-11T05:13:20.490994+0000","last_active":"2024-09-11T05:13:20.490994+0000","last_peered":"2024-09-11T05:13:19.500708+0000","last_clean":"2024-09-11T05:12:42.358674+0000","last_became_active":"2024-09-11T05:13:19.499107+0000","last_became_peered":"2024-09-11T05:13:19.499107+0000","last_unstale":"2024-09-11T05:13:19.500708+0000","last_undegraded":"2024-09-11T05:13:19.486113+0000","last_fullsized":"2024-09-11T05:13:19.500708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":59,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.502180+0000","last_change":"2024-09-11T05:13:20.494853+0000","last_active":"2024-09-11T05:13:20.494853+0000","last_peered":"2024-09-11T05:13:19.502180+0000","last_clean":"2024-09-11T05:13:19.502180+0000","last_became_active":"2024-09-11T05:13:19.499389+0000","last_became_peered":"2024-09-11T05:13:19.499389+0000","last_unstale":"2024-09-11T05:13:19.502180+0000","last_undegraded":"2024-09-11T05:13:19.502180+0000","last_fullsized":"2024-09-11T05:13:19.502180+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":52,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.495675+0000","last_change":"2024-09-11T05:13:20.488850+0000","last_active":"2024-09-11T05:13:20.488850+0000","last_peered":"2024-09-11T05:13:19.495675+0000","last_clean":"2024-09-11T05:12:42.979390+0000","last_became_active":"2024-09-11T05:13:19.494820+0000","last_became_peered":"2024-09-11T05:13:19.494820+0000","last_unstale":"2024-09-11T05:13:19.495675+0000","last_undegraded":"2024-09-11T05:13:19.486670+0000","last_fullsized":"2024-09-11T05:13:19.495675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'1","reported_seq":51,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.490122+0000","last_change":"2024-09-11T05:13:20.498766+0000","last_active":"2024-09-11T05:13:20.498766+0000","last_peered":"2024-09-11T05:13:19.490122+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:13:02.466160+0000","last_became_peered":"2024-09-11T05:13:02.466160+0000","last_unstale":"2024-09-11T05:13:19.490122+0000","last_undegraded":"2024-09-11T05:13:02.460401+0000","last_fullsized":"2024-09-11T05:13:02.460103+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.489693+0000","last_change":"2024-09-11T05:13:20.490971+0000","last_active":"2024-09-11T05:13:20.490971+0000","last_peered":"2024-09-11T05:13:19.489693+0000","last_clean":"2024-09-11T05:12:35.196045+0000","last_became_active":"2024-09-11T05:13:02.464357+0000","last_became_peered":"2024-09-11T05:13:02.464357+0000","last_unstale":"2024-09-11T05:13:19.489693+0000","last_undegraded":"2024-09-11T05:13:19.489693+0000","last_fullsized":"2024-09-11T05:13:02.459737+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"22'5","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.489325+0000","last_change":"2024-09-11T05:13:20.500090+0000","last_active":"2024-09-11T05:13:20.500090+0000","last_peered":"2024-09-11T05:13:19.489325+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:13:02.467735+0000","last_became_peered":"2024-09-11T05:13:02.467735+0000","last_unstale":"2024-09-11T05:13:19.489325+0000","last_undegraded":"2024-09-11T05:13:02.460769+0000","last_fullsized":"2024-09-11T05:13:02.460446+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.502897+0000","last_change":"2024-09-11T05:13:20.489403+0000","last_active":"2024-09-11T05:13:20.489403+0000","last_peered":"2024-09-11T05:13:19.502897+0000","last_clean":"2024-09-11T05:13:19.502897+0000","last_became_active":"2024-09-11T05:13:19.499404+0000","last_became_peered":"2024-09-11T05:13:19.499404+0000","last_unstale":"2024-09-11T05:13:19.502897+0000","last_undegraded":"2024-09-11T05:13:19.502897+0000","last_fullsized":"2024-09-11T05:13:19.502897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"22'2","reported_seq":52,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.489266+0000","last_change":"2024-09-11T05:13:20.498731+0000","last_active":"2024-09-11T05:13:20.498731+0000","last_peered":"2024-09-11T05:13:19.489266+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:13:02.464917+0000","last_became_peered":"2024-09-11T05:13:02.464917+0000","last_unstale":"2024-09-11T05:13:19.489266+0000","last_undegraded":"2024-09-11T05:13:02.459925+0000","last_fullsized":"2024-09-11T05:13:02.459610+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.488639+0000","last_change":"2024-09-11T05:13:20.498859+0000","last_active":"2024-09-11T05:13:20.498859+0000","last_peered":"2024-09-11T05:13:19.488639+0000","last_clean":"2024-09-11T05:12:35.198361+0000","last_became_active":"2024-09-11T05:13:02.464680+0000","last_became_peered":"2024-09-11T05:13:02.464680+0000","last_unstale":"2024-09-11T05:13:19.488639+0000","last_undegraded":"2024-09-11T05:13:19.488639+0000","last_fullsized":"2024-09-11T05:13:02.459421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.497261+0000","last_change":"2024-09-11T05:13:20.488765+0000","last_active":"2024-09-11T05:13:20.488765+0000","last_peered":"2024-09-11T05:13:19.497261+0000","last_clean":"2024-09-11T05:13:19.497261+0000","last_became_active":"2024-09-11T05:13:19.494452+0000","last_became_peered":"2024-09-11T05:13:19.494452+0000","last_unstale":"2024-09-11T05:13:19.497261+0000","last_undegraded":"2024-09-11T05:13:19.497261+0000","last_fullsized":"2024-09-11T05:13:19.497261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"22'9","reported_seq":61,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T05:13:19.489313+0000","last_change":"2024-09-11T05:13:20.491471+0000","last_active":"2024-09-11T05:13:20.491471+0000","last_peered":"2024-09-11T05:13:19.489313+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:13:02.462992+0000","last_became_peered":"2024-09-11T05:13:02.462992+0000","last_unstale":"2024-09-11T05:13:19.489313+0000","last_undegraded":"2024-09-11T05:13:02.459192+0000","last_fullsized":"2024-09-11T05:13:02.458822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":278528,"data_stored":86557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28176043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215112,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":316,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":323584,"data_stored":92746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:23.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:13:23.434 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:23.434 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726031597,"num_in_osds":2,"osd_in_since":1726031600,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56950784,"bytes_avail":193216577536,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T05:13:13.354801+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:13:23.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '40251081-3433-4f55-b0e4-ce0789fd19fd', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 81, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726031597, 'num_in_osds': 2, 'osd_in_since': 1726031600, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56950784, 'bytes_avail': 193216577536, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T05:12:02:199900+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T05:13:13.354801+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T05:13:26.452 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:26.756 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:26.756 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:26.772 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-11T05:13:25.358935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55640,"kb_used_data":612,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688040,"statfs":{"total":193273528320,"available":193216552960,"internally_reserved":0,"allocated":626688,"data_stored":188274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":56286550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-21,"num_objects_misplaced":0,"num_objects_unfound":-9,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.880039"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":53,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495634+0000","last_change":"2024-09-11T05:13:21.515668+0000","last_active":"2024-09-11T05:13:22.495634+0000","last_peered":"2024-09-11T05:13:22.495634+0000","last_clean":"2024-09-11T05:13:22.495634+0000","last_became_active":"2024-09-11T05:13:21.514919+0000","last_became_peered":"2024-09-11T05:13:21.514919+0000","last_unstale":"2024-09-11T05:13:22.495634+0000","last_undegraded":"2024-09-11T05:13:22.495634+0000","last_fullsized":"2024-09-11T05:13:22.495634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.714157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'4","reported_seq":46,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.495347+0000","last_change":"2024-09-11T05:13:20.498602+0000","last_active":"2024-09-11T05:13:20.498602+0000","last_peered":"2024-09-11T05:13:19.495347+0000","last_clean":"2024-09-11T05:12:42.606590+0000","last_became_active":"2024-09-11T05:13:19.494550+0000","last_became_peered":"2024-09-11T05:13:19.494550+0000","last_unstale":"2024-09-11T05:13:19.495347+0000","last_undegraded":"2024-09-11T05:13:19.486154+0000","last_fullsized":"2024-09-11T05:13:19.495347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":62,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.500074+0000","last_change":"2024-09-11T05:13:20.491369+0000","last_active":"2024-09-11T05:13:20.491369+0000","last_peered":"2024-09-11T05:13:19.500074+0000","last_clean":"2024-09-11T05:12:43.483894+0000","last_became_active":"2024-09-11T05:13:19.499437+0000","last_became_peered":"2024-09-11T05:13:19.499437+0000","last_unstale":"2024-09-11T05:13:19.500074+0000","last_undegraded":"2024-09-11T05:13:19.487419+0000","last_fullsized":"2024-09-11T05:13:19.500074+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.496251+0000","last_change":"2024-09-11T05:13:20.497118+0000","last_active":"2024-09-11T05:13:20.497118+0000","last_peered":"2024-09-11T05:13:19.496251+0000","last_clean":"2024-09-11T05:13:19.496251+0000","last_became_active":"2024-09-11T05:13:19.493075+0000","last_became_peered":"2024-09-11T05:13:19.493075+0000","last_unstale":"2024-09-11T05:13:19.496251+0000","last_undegraded":"2024-09-11T05:13:19.496251+0000","last_fullsized":"2024-09-11T05:13:19.496251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":66,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494668+0000","last_change":"2024-09-11T05:13:21.517179+0000","last_active":"2024-09-11T05:13:22.494668+0000","last_peered":"2024-09-11T05:13:22.494668+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:13:22.494668+0000","last_undegraded":"2024-09-11T05:13:21.496359+0000","last_fullsized":"2024-09-11T05:13:22.494668+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"22'3","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493977+0000","last_change":"2024-09-11T05:13:21.514194+0000","last_active":"2024-09-11T05:13:22.493977+0000","last_peered":"2024-09-11T05:13:22.493977+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:13:21.512885+0000","last_became_peered":"2024-09-11T05:13:21.512885+0000","last_unstale":"2024-09-11T05:13:22.493977+0000","last_undegraded":"2024-09-11T05:13:21.492320+0000","last_fullsized":"2024-09-11T05:13:22.493977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"22'4","reported_seq":63,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494011+0000","last_change":"2024-09-11T05:13:21.511966+0000","last_active":"2024-09-11T05:13:22.494011+0000","last_peered":"2024-09-11T05:13:22.494011+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:13:21.511332+0000","last_became_peered":"2024-09-11T05:13:21.511332+0000","last_unstale":"2024-09-11T05:13:22.494011+0000","last_undegraded":"2024-09-11T05:13:21.492627+0000","last_fullsized":"2024-09-11T05:13:22.494011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'1","reported_seq":64,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-11T05:13:22.493892+0000","last_change":"2024-09-11T05:13:22.493892+0000","last_active":"2024-09-11T05:13:22.493892+0000","last_peered":"2024-09-11T05:13:22.493892+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:13:22.493892+0000","last_undegraded":"2024-09-11T05:13:22.493892+0000","last_fullsized":"2024-09-11T05:13:22.493892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.493688+0000","last_change":"2024-09-11T05:13:21.508556+0000","last_active":"2024-09-11T05:13:22.493688+0000","last_peered":"2024-09-11T05:13:22.493688+0000","last_clean":"2024-09-11T05:13:22.493688+0000","last_became_active":"2024-09-11T05:13:21.507784+0000","last_became_peered":"2024-09-11T05:13:21.507784+0000","last_unstale":"2024-09-11T05:13:22.493688+0000","last_undegraded":"2024-09-11T05:13:22.493688+0000","last_fullsized":"2024-09-11T05:13:22.493688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:10.209768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"22'5","reported_seq":64,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.500015+0000","last_change":"2024-09-11T05:13:20.489110+0000","last_active":"2024-09-11T05:13:20.489110+0000","last_peered":"2024-09-11T05:13:19.500015+0000","last_clean":"2024-09-11T05:12:43.565164+0000","last_became_active":"2024-09-11T05:13:19.499294+0000","last_became_peered":"2024-09-11T05:13:19.499294+0000","last_unstale":"2024-09-11T05:13:19.500015+0000","last_undegraded":"2024-09-11T05:13:19.487494+0000","last_fullsized":"2024-09-11T05:13:19.500015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.499766+0000","last_change":"2024-09-11T05:13:20.489707+0000","last_active":"2024-09-11T05:13:20.489707+0000","last_peered":"2024-09-11T05:13:19.499766+0000","last_clean":"2024-09-11T05:13:19.499766+0000","last_became_active":"2024-09-11T05:13:19.495911+0000","last_became_peered":"2024-09-11T05:13:19.495911+0000","last_unstale":"2024-09-11T05:13:19.499766+0000","last_undegraded":"2024-09-11T05:13:19.499766+0000","last_fullsized":"2024-09-11T05:13:19.499766+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'6","reported_seq":58,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494187+0000","last_change":"2024-09-11T05:13:21.514035+0000","last_active":"2024-09-11T05:13:22.494187+0000","last_peered":"2024-09-11T05:13:22.494187+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:13:21.511727+0000","last_became_peered":"2024-09-11T05:13:21.511727+0000","last_unstale":"2024-09-11T05:13:22.494187+0000","last_undegraded":"2024-09-11T05:13:21.493049+0000","last_fullsized":"2024-09-11T05:13:22.494187+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493699+0000","last_change":"2024-09-11T05:13:21.513836+0000","last_active":"2024-09-11T05:13:22.493699+0000","last_peered":"2024-09-11T05:13:22.493699+0000","last_clean":"2024-09-11T05:12:55.450586+0000","last_became_active":"2024-09-11T05:13:21.506029+0000","last_became_peered":"2024-09-11T05:13:21.506029+0000","last_unstale":"2024-09-11T05:13:22.493699+0000","last_undegraded":"2024-09-11T05:13:21.491804+0000","last_fullsized":"2024-09-11T05:13:22.493699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'3","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.500708+0000","last_change":"2024-09-11T05:13:20.490994+0000","last_active":"2024-09-11T05:13:20.490994+0000","last_peered":"2024-09-11T05:13:19.500708+0000","last_clean":"2024-09-11T05:12:42.358674+0000","last_became_active":"2024-09-11T05:13:19.499107+0000","last_became_peered":"2024-09-11T05:13:19.499107+0000","last_unstale":"2024-09-11T05:13:19.500708+0000","last_undegraded":"2024-09-11T05:13:19.486113+0000","last_fullsized":"2024-09-11T05:13:19.500708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":59,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.502180+0000","last_change":"2024-09-11T05:13:20.494853+0000","last_active":"2024-09-11T05:13:20.494853+0000","last_peered":"2024-09-11T05:13:19.502180+0000","last_clean":"2024-09-11T05:13:19.502180+0000","last_became_active":"2024-09-11T05:13:19.499389+0000","last_became_peered":"2024-09-11T05:13:19.499389+0000","last_unstale":"2024-09-11T05:13:19.502180+0000","last_undegraded":"2024-09-11T05:13:19.502180+0000","last_fullsized":"2024-09-11T05:13:19.502180+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":52,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.495675+0000","last_change":"2024-09-11T05:13:20.488850+0000","last_active":"2024-09-11T05:13:20.488850+0000","last_peered":"2024-09-11T05:13:19.495675+0000","last_clean":"2024-09-11T05:12:42.979390+0000","last_became_active":"2024-09-11T05:13:19.494820+0000","last_became_peered":"2024-09-11T05:13:19.494820+0000","last_unstale":"2024-09-11T05:13:19.495675+0000","last_undegraded":"2024-09-11T05:13:19.486670+0000","last_fullsized":"2024-09-11T05:13:19.495675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'1","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.495413+0000","last_change":"2024-09-11T05:13:21.512469+0000","last_active":"2024-09-11T05:13:22.495413+0000","last_peered":"2024-09-11T05:13:22.495413+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:13:21.511812+0000","last_became_peered":"2024-09-11T05:13:21.511812+0000","last_unstale":"2024-09-11T05:13:22.495413+0000","last_undegraded":"2024-09-11T05:13:21.496652+0000","last_fullsized":"2024-09-11T05:13:22.495413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.494954+0000","last_change":"2024-09-11T05:13:21.512849+0000","last_active":"2024-09-11T05:13:22.494954+0000","last_peered":"2024-09-11T05:13:22.494954+0000","last_clean":"2024-09-11T05:13:22.494954+0000","last_became_active":"2024-09-11T05:13:21.512089+0000","last_became_peered":"2024-09-11T05:13:21.512089+0000","last_unstale":"2024-09-11T05:13:22.494954+0000","last_undegraded":"2024-09-11T05:13:22.494954+0000","last_fullsized":"2024-09-11T05:13:22.494954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:09:25.087655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"22'5","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.495243+0000","last_change":"2024-09-11T05:13:21.516438+0000","last_active":"2024-09-11T05:13:22.495243+0000","last_peered":"2024-09-11T05:13:22.495243+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:13:21.515407+0000","last_became_peered":"2024-09-11T05:13:21.515407+0000","last_unstale":"2024-09-11T05:13:22.495243+0000","last_undegraded":"2024-09-11T05:13:21.496548+0000","last_fullsized":"2024-09-11T05:13:22.495243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.502897+0000","last_change":"2024-09-11T05:13:20.489403+0000","last_active":"2024-09-11T05:13:20.489403+0000","last_peered":"2024-09-11T05:13:19.502897+0000","last_clean":"2024-09-11T05:13:19.502897+0000","last_became_active":"2024-09-11T05:13:19.499404+0000","last_became_peered":"2024-09-11T05:13:19.499404+0000","last_unstale":"2024-09-11T05:13:19.502897+0000","last_undegraded":"2024-09-11T05:13:19.502897+0000","last_fullsized":"2024-09-11T05:13:19.502897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"22'2","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.495156+0000","last_change":"2024-09-11T05:13:21.516118+0000","last_active":"2024-09-11T05:13:22.495156+0000","last_peered":"2024-09-11T05:13:22.495156+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:13:21.515226+0000","last_became_peered":"2024-09-11T05:13:21.515226+0000","last_unstale":"2024-09-11T05:13:22.495156+0000","last_undegraded":"2024-09-11T05:13:21.494389+0000","last_fullsized":"2024-09-11T05:13:22.495156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495153+0000","last_change":"2024-09-11T05:13:21.511859+0000","last_active":"2024-09-11T05:13:22.495153+0000","last_peered":"2024-09-11T05:13:22.495153+0000","last_clean":"2024-09-11T05:13:22.495153+0000","last_became_active":"2024-09-11T05:13:21.511091+0000","last_became_peered":"2024-09-11T05:13:21.511091+0000","last_unstale":"2024-09-11T05:13:22.495153+0000","last_undegraded":"2024-09-11T05:13:22.495153+0000","last_fullsized":"2024-09-11T05:13:22.495153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:05.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T05:13:19.497261+0000","last_change":"2024-09-11T05:13:20.488765+0000","last_active":"2024-09-11T05:13:20.488765+0000","last_peered":"2024-09-11T05:13:19.497261+0000","last_clean":"2024-09-11T05:13:19.497261+0000","last_became_active":"2024-09-11T05:13:19.494452+0000","last_became_peered":"2024-09-11T05:13:19.494452+0000","last_unstale":"2024-09-11T05:13:19.497261+0000","last_undegraded":"2024-09-11T05:13:19.497261+0000","last_fullsized":"2024-09-11T05:13:19.497261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"22'9","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494953+0000","last_change":"2024-09-11T05:13:21.513883+0000","last_active":"2024-09-11T05:13:22.494953+0000","last_peered":"2024-09-11T05:13:22.494953+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:13:21.512960+0000","last_became_peered":"2024-09-11T05:13:21.512960+0000","last_unstale":"2024-09-11T05:13:22.494953+0000","last_undegraded":"2024-09-11T05:13:21.493149+0000","last_fullsized":"2024-09-11T05:13:22.494953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":278528,"data_stored":86557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28176043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215113,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":348160,"data_stored":101717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:26.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:13:27.112 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:27.112 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"18/30 objects unfound (60.000%)","count":18},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36/60 objects degraded (60.000%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":84,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726031597,"num_in_osds":2,"osd_in_since":1726031600,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"remapped+peering","count":10},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+clean","count":4},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56975360,"bytes_avail":193216552960,"bytes_total":193273528320,"inactive_pgs_ratio":0.4166666567325592,"degraded_objects":36,"degraded_total":60,"degraded_ratio":0.59999999999999998,"unfound_objects":18,"unfound_total":30,"unfound_ratio":0.59999999999999998},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T05:13:13.354801+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:13:27.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '40251081-3433-4f55-b0e4-ce0789fd19fd', 'health': {'status': 'HEALTH_WARN', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '18/30 objects unfound (60.000%)', 'count': 18}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 36/60 objects degraded (60.000%), 9 pgs degraded', 'count': 9}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 84, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726031597, 'num_in_osds': 2, 'osd_in_since': 1726031600, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'remapped+peering', 'count': 10}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+clean', 'count': 4}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56975360, 'bytes_avail': 193216552960, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 0.4166666567325592, 'degraded_objects': 36, 'degraded_total': 60, 'degraded_ratio': 0.6, 'unfound_objects': 18, 'unfound_total': 30, 'unfound_ratio': 0.6}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T05:12:02:199900+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T05:13:13.354801+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T05:13:27.465 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:13:27.462+0000 7f9b77f07640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-11T05:13:30.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:30.429 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:30.430 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:30.446 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-11T05:13:29.360554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55668,"kb_used_data":636,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688012,"statfs":{"total":193273528320,"available":193216524288,"internally_reserved":0,"allocated":651264,"data_stored":197245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":56286550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002655"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":53,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495634+0000","last_change":"2024-09-11T05:13:21.515668+0000","last_active":"2024-09-11T05:13:22.495634+0000","last_peered":"2024-09-11T05:13:22.495634+0000","last_clean":"2024-09-11T05:13:22.495634+0000","last_became_active":"2024-09-11T05:13:21.514919+0000","last_became_peered":"2024-09-11T05:13:21.514919+0000","last_unstale":"2024-09-11T05:13:22.495634+0000","last_undegraded":"2024-09-11T05:13:22.495634+0000","last_fullsized":"2024-09-11T05:13:22.495634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.714157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'4","reported_seq":50,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-11T05:13:22.501587+0000","last_change":"2024-09-11T05:13:22.501760+0000","last_active":"2024-09-11T05:13:22.501587+0000","last_peered":"2024-09-11T05:13:22.501587+0000","last_clean":"2024-09-11T05:12:42.606590+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:13:22.501587+0000","last_undegraded":"2024-09-11T05:13:22.489931+0000","last_fullsized":"2024-09-11T05:13:22.501587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":67,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T05:13:22.501282+0000","last_change":"2024-09-11T05:13:25.436303+0000","last_active":"2024-09-11T05:13:22.501282+0000","last_peered":"2024-09-11T05:13:22.501282+0000","last_clean":"2024-09-11T05:12:43.483894+0000","last_became_active":"2024-09-11T05:13:22.497399+0000","last_became_peered":"2024-09-11T05:13:22.497399+0000","last_unstale":"2024-09-11T05:13:22.501282+0000","last_undegraded":"2024-09-11T05:13:22.501282+0000","last_fullsized":"2024-09-11T05:13:22.501282+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.498752+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:13:22.498752+0000","last_peered":"2024-09-11T05:13:22.498752+0000","last_clean":"2024-09-11T05:13:22.498752+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:13:22.498752+0000","last_undegraded":"2024-09-11T05:13:22.498752+0000","last_fullsized":"2024-09-11T05:13:22.498752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"22'5","reported_seq":66,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494668+0000","last_change":"2024-09-11T05:13:21.517179+0000","last_active":"2024-09-11T05:13:22.494668+0000","last_peered":"2024-09-11T05:13:22.494668+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:13:22.494668+0000","last_undegraded":"2024-09-11T05:13:21.496359+0000","last_fullsized":"2024-09-11T05:13:22.494668+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"22'3","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493977+0000","last_change":"2024-09-11T05:13:21.514194+0000","last_active":"2024-09-11T05:13:22.493977+0000","last_peered":"2024-09-11T05:13:22.493977+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:13:21.512885+0000","last_became_peered":"2024-09-11T05:13:21.512885+0000","last_unstale":"2024-09-11T05:13:22.493977+0000","last_undegraded":"2024-09-11T05:13:21.492320+0000","last_fullsized":"2024-09-11T05:13:22.493977+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"22'4","reported_seq":63,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494011+0000","last_change":"2024-09-11T05:13:21.511966+0000","last_active":"2024-09-11T05:13:22.494011+0000","last_peered":"2024-09-11T05:13:22.494011+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:13:21.511332+0000","last_became_peered":"2024-09-11T05:13:21.511332+0000","last_unstale":"2024-09-11T05:13:22.494011+0000","last_undegraded":"2024-09-11T05:13:21.492627+0000","last_fullsized":"2024-09-11T05:13:22.494011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'1","reported_seq":64,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-11T05:13:22.493892+0000","last_change":"2024-09-11T05:13:22.493892+0000","last_active":"2024-09-11T05:13:22.493892+0000","last_peered":"2024-09-11T05:13:22.493892+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:13:22.493892+0000","last_undegraded":"2024-09-11T05:13:22.493892+0000","last_fullsized":"2024-09-11T05:13:22.493892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.493688+0000","last_change":"2024-09-11T05:13:21.508556+0000","last_active":"2024-09-11T05:13:22.493688+0000","last_peered":"2024-09-11T05:13:22.493688+0000","last_clean":"2024-09-11T05:13:22.493688+0000","last_became_active":"2024-09-11T05:13:21.507784+0000","last_became_peered":"2024-09-11T05:13:21.507784+0000","last_unstale":"2024-09-11T05:13:22.493688+0000","last_undegraded":"2024-09-11T05:13:22.493688+0000","last_fullsized":"2024-09-11T05:13:22.493688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:10.209768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"22'5","reported_seq":69,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T05:13:22.500791+0000","last_change":"2024-09-11T05:13:25.436201+0000","last_active":"2024-09-11T05:13:22.500791+0000","last_peered":"2024-09-11T05:13:22.500791+0000","last_clean":"2024-09-11T05:12:43.565164+0000","last_became_active":"2024-09-11T05:13:22.496902+0000","last_became_peered":"2024-09-11T05:13:22.496902+0000","last_unstale":"2024-09-11T05:13:22.500791+0000","last_undegraded":"2024-09-11T05:13:22.500791+0000","last_fullsized":"2024-09-11T05:13:22.500791+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497168+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:13:22.497168+0000","last_peered":"2024-09-11T05:13:22.497168+0000","last_clean":"2024-09-11T05:13:22.497168+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:13:22.497168+0000","last_undegraded":"2024-09-11T05:13:22.497168+0000","last_fullsized":"2024-09-11T05:13:22.497168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'6","reported_seq":58,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494187+0000","last_change":"2024-09-11T05:13:21.514035+0000","last_active":"2024-09-11T05:13:22.494187+0000","last_peered":"2024-09-11T05:13:22.494187+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:13:21.511727+0000","last_became_peered":"2024-09-11T05:13:21.511727+0000","last_unstale":"2024-09-11T05:13:22.494187+0000","last_undegraded":"2024-09-11T05:13:21.493049+0000","last_fullsized":"2024-09-11T05:13:22.494187+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493699+0000","last_change":"2024-09-11T05:13:21.513836+0000","last_active":"2024-09-11T05:13:22.493699+0000","last_peered":"2024-09-11T05:13:22.493699+0000","last_clean":"2024-09-11T05:12:55.450586+0000","last_became_active":"2024-09-11T05:13:21.506029+0000","last_became_peered":"2024-09-11T05:13:21.506029+0000","last_unstale":"2024-09-11T05:13:22.493699+0000","last_undegraded":"2024-09-11T05:13:21.491804+0000","last_fullsized":"2024-09-11T05:13:22.493699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'3","reported_seq":55,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-11T05:13:22.496966+0000","last_change":"2024-09-11T05:13:22.497240+0000","last_active":"2024-09-11T05:13:22.496966+0000","last_peered":"2024-09-11T05:13:22.496966+0000","last_clean":"2024-09-11T05:12:42.358674+0000","last_became_active":"2024-09-11T05:13:22.495923+0000","last_became_peered":"2024-09-11T05:13:22.495923+0000","last_unstale":"2024-09-11T05:13:22.496966+0000","last_undegraded":"2024-09-11T05:13:22.488424+0000","last_fullsized":"2024-09-11T05:13:22.496966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505097+0000","last_change":"2024-09-11T05:13:22.497695+0000","last_active":"2024-09-11T05:13:22.505097+0000","last_peered":"2024-09-11T05:13:22.505097+0000","last_clean":"2024-09-11T05:13:22.505097+0000","last_became_active":"2024-09-11T05:13:22.497288+0000","last_became_peered":"2024-09-11T05:13:22.497288+0000","last_unstale":"2024-09-11T05:13:22.505097+0000","last_undegraded":"2024-09-11T05:13:22.505097+0000","last_fullsized":"2024-09-11T05:13:22.505097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:56.694811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":56,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-11T05:13:22.499986+0000","last_change":"2024-09-11T05:13:22.500173+0000","last_active":"2024-09-11T05:13:22.499986+0000","last_peered":"2024-09-11T05:13:22.499986+0000","last_clean":"2024-09-11T05:12:42.979390+0000","last_became_active":"2024-09-11T05:13:22.496829+0000","last_became_peered":"2024-09-11T05:13:22.496829+0000","last_unstale":"2024-09-11T05:13:22.499986+0000","last_undegraded":"2024-09-11T05:13:22.488959+0000","last_fullsized":"2024-09-11T05:13:22.499986+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"22'1","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.495413+0000","last_change":"2024-09-11T05:13:21.512469+0000","last_active":"2024-09-11T05:13:22.495413+0000","last_peered":"2024-09-11T05:13:22.495413+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:13:21.511812+0000","last_became_peered":"2024-09-11T05:13:21.511812+0000","last_unstale":"2024-09-11T05:13:22.495413+0000","last_undegraded":"2024-09-11T05:13:21.496652+0000","last_fullsized":"2024-09-11T05:13:22.495413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.494954+0000","last_change":"2024-09-11T05:13:21.512849+0000","last_active":"2024-09-11T05:13:22.494954+0000","last_peered":"2024-09-11T05:13:22.494954+0000","last_clean":"2024-09-11T05:13:22.494954+0000","last_became_active":"2024-09-11T05:13:21.512089+0000","last_became_peered":"2024-09-11T05:13:21.512089+0000","last_unstale":"2024-09-11T05:13:22.494954+0000","last_undegraded":"2024-09-11T05:13:22.494954+0000","last_fullsized":"2024-09-11T05:13:22.494954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:09:25.087655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"22'5","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.495243+0000","last_change":"2024-09-11T05:13:21.516438+0000","last_active":"2024-09-11T05:13:22.495243+0000","last_peered":"2024-09-11T05:13:22.495243+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:13:21.515407+0000","last_became_peered":"2024-09-11T05:13:21.515407+0000","last_unstale":"2024-09-11T05:13:22.495243+0000","last_undegraded":"2024-09-11T05:13:21.496548+0000","last_fullsized":"2024-09-11T05:13:22.495243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497841+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:13:22.497841+0000","last_peered":"2024-09-11T05:13:22.497841+0000","last_clean":"2024-09-11T05:13:22.497841+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:13:22.497841+0000","last_undegraded":"2024-09-11T05:13:22.497841+0000","last_fullsized":"2024-09-11T05:13:22.497841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"22'2","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.495156+0000","last_change":"2024-09-11T05:13:21.516118+0000","last_active":"2024-09-11T05:13:22.495156+0000","last_peered":"2024-09-11T05:13:22.495156+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:13:21.515226+0000","last_became_peered":"2024-09-11T05:13:21.515226+0000","last_unstale":"2024-09-11T05:13:22.495156+0000","last_undegraded":"2024-09-11T05:13:21.494389+0000","last_fullsized":"2024-09-11T05:13:22.495156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495153+0000","last_change":"2024-09-11T05:13:21.511859+0000","last_active":"2024-09-11T05:13:22.495153+0000","last_peered":"2024-09-11T05:13:22.495153+0000","last_clean":"2024-09-11T05:13:22.495153+0000","last_became_active":"2024-09-11T05:13:21.511091+0000","last_became_peered":"2024-09-11T05:13:21.511091+0000","last_unstale":"2024-09-11T05:13:22.495153+0000","last_undegraded":"2024-09-11T05:13:22.495153+0000","last_fullsized":"2024-09-11T05:13:22.495153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:05.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505152+0000","last_change":"2024-09-11T05:13:22.498193+0000","last_active":"2024-09-11T05:13:22.505152+0000","last_peered":"2024-09-11T05:13:22.505152+0000","last_clean":"2024-09-11T05:13:22.505152+0000","last_became_active":"2024-09-11T05:13:22.498019+0000","last_became_peered":"2024-09-11T05:13:22.498019+0000","last_unstale":"2024-09-11T05:13:22.505152+0000","last_undegraded":"2024-09-11T05:13:22.505152+0000","last_fullsized":"2024-09-11T05:13:22.505152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:24.352250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"22'9","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.494953+0000","last_change":"2024-09-11T05:13:21.513883+0000","last_active":"2024-09-11T05:13:22.494953+0000","last_peered":"2024-09-11T05:13:22.494953+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:13:21.512960+0000","last_became_peered":"2024-09-11T05:13:21.512960+0000","last_unstale":"2024-09-11T05:13:22.494953+0000","last_undegraded":"2024-09-11T05:13:21.493149+0000","last_fullsized":"2024-09-11T05:13:22.494953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986180,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":296,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":303104,"data_stored":95528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28176043},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215114,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":348160,"data_stored":101717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:30.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:13:30.796 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:30.796 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":88,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726031597,"num_in_osds":2,"osd_in_since":1726031600,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":57004032,"bytes_avail":193216524288,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T05:13:13.354801+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:13:30.813 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '40251081-3433-4f55-b0e4-ce0789fd19fd', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 88, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726031597, 'num_in_osds': 2, 'osd_in_since': 1726031600, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 57004032, 'bytes_avail': 193216524288, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T05:12:02:199900+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T05:13:13.354801+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T05:13:30.813 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-11T05:13:30.814 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T05:13:30.814 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T05:13:30.959 INFO:teuthology.orchestra.run.smithi133.stdout:133143986182 2024-09-11T05:13:30.959 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:13:30.964 INFO:teuthology.orchestra.run.smithi133.stdout:103079215115 2024-09-11T05:13:30.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:13:31.240 INFO:teuthology.orchestra.run.smithi133.stdout:133143986180 2024-09-11T05:13:31.257 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986182 got 133143986180 for osd.2 2024-09-11T05:13:31.285 INFO:teuthology.orchestra.run.smithi133.stdout:103079215114 2024-09-11T05:13:31.302 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215115 got 103079215114 for osd.0 2024-09-11T05:13:32.258 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:13:32.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:13:32.573 INFO:teuthology.orchestra.run.smithi133.stdout:133143986182 2024-09-11T05:13:32.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986182 got 133143986182 for osd.2 2024-09-11T05:13:32.592 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:32.600 INFO:teuthology.orchestra.run.smithi133.stdout:103079215115 2024-09-11T05:13:32.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215115 got 103079215115 for osd.0 2024-09-11T05:13:32.618 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:32.618 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:13:33.000 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:33.000 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":90,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726031597,"num_in_osds":2,"osd_in_since":1726031600,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":57004032,"bytes_avail":193216524288,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T05:13:13.354801+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:13:33.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '40251081-3433-4f55-b0e4-ce0789fd19fd', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 90, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726031597, 'num_in_osds': 2, 'osd_in_since': 1726031600, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 57004032, 'bytes_avail': 193216524288, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T05:12:02:199900+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T05:13:13.354801+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T05:13:33.018 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-09-11T05:13:33.018 DEBUG:teuthology.orchestra.run.smithi133:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-09-11T05:13:33.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:hints = 1 2024-09-11T05:13:33.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-11T05:13:33.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Object prefix: benchmark_data_smithi133_33840 2024-09-11T05:13:33.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:13:33.063 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 0 0 0 0 0 0 - 0 2024-09-11T05:13:34.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 1 20 2840 2820 10.9854 11.0156 0.00894706 0.00698036 2024-09-11T05:13:35.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 2 20 5517 5497 10.7211 10.457 0.0072085 0.00726019 2024-09-11T05:13:36.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 3 20 8059 8039 10.4573 9.92969 0.00916634 0.00745558 2024-09-11T05:13:37.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 4 20 10490 10470 10.217 9.49609 0.00666779 0.00763612 2024-09-11T05:13:38.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 5 20 12770 12750 9.95483 8.90625 0.00830026 0.00783852 2024-09-11T05:13:39.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 6 20 14886 14866 9.67331 8.26562 0.00484624 0.00805662 2024-09-11T05:13:40.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 7 20 16961 16941 9.44933 8.10547 0.00988453 0.00825882 2024-09-11T05:13:41.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 8 20 18936 18916 9.23251 7.71484 0.0104865 0.00845256 2024-09-11T05:13:42.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 9 20 20854 20834 9.0391 7.49219 0.00851165 0.00863389 2024-09-11T05:13:43.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:43.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 10 20 22584 22564 8.81098 6.75781 0.0119153 0.00885598 2024-09-11T05:13:43.505 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:43.505 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:43.534 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-11T05:13:43.364899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":61580175,"num_objects":15060,"num_object_clones":0,"num_object_copies":30120,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":15060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15087,"num_write_kb":60168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15088,"ondisk_log_size":15088,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":249644,"kb_used_data":123852,"kb_used_omap":8,"kb_used_meta":125751,"kb_avail":188494036,"statfs":{"total":193273528320,"available":193017892864,"internally_reserved":0,"allocated":126824448,"data_stored":126370429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":128769366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,7,4,9,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61562880,"num_objects":15030,"num_object_clones":0,"num_object_copies":30060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15030,"num_write_kb":60120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003541"},"pg_stats":[{"pgid":"2.f","version":"36'855","reported_seq":908,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:38.439501+0000","last_change":"2024-09-11T05:13:21.515668+0000","last_active":"2024-09-11T05:13:38.439501+0000","last_peered":"2024-09-11T05:13:38.439501+0000","last_clean":"2024-09-11T05:13:38.439501+0000","last_became_active":"2024-09-11T05:13:21.514919+0000","last_became_peered":"2024-09-11T05:13:21.514919+0000","last_unstale":"2024-09-11T05:13:38.439501+0000","last_undegraded":"2024-09-11T05:13:38.439501+0000","last_fullsized":"2024-09-11T05:13:38.439501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.714157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502080,"num_objects":855,"num_object_clones":0,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":855,"num_write_kb":3420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'1101","reported_seq":1147,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T05:13:40.438575+0000","last_change":"2024-09-11T05:13:33.083057+0000","last_active":"2024-09-11T05:13:40.438575+0000","last_peered":"2024-09-11T05:13:40.438575+0000","last_clean":"2024-09-11T05:12:42.606590+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:13:40.438575+0000","last_undegraded":"2024-09-11T05:13:22.501760+0000","last_fullsized":"2024-09-11T05:13:40.438575+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4494546,"num_objects":1099,"num_object_clones":0,"num_object_copies":2198,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1101,"num_write_kb":4392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1099}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'1051","reported_seq":1115,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T05:13:40.428517+0000","last_change":"2024-09-11T05:13:25.436303+0000","last_active":"2024-09-11T05:13:40.428517+0000","last_peered":"2024-09-11T05:13:40.428517+0000","last_clean":"2024-09-11T05:12:43.483894+0000","last_became_active":"2024-09-11T05:13:22.497399+0000","last_became_peered":"2024-09-11T05:13:22.497399+0000","last_unstale":"2024-09-11T05:13:40.428517+0000","last_undegraded":"2024-09-11T05:13:22.501282+0000","last_fullsized":"2024-09-11T05:13:40.428517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1051,"log_dups_size":0,"ondisk_log_size":1051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4293225,"num_objects":1049,"num_object_clones":0,"num_object_copies":2098,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1051,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'1129","reported_seq":1181,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:40.424020+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:13:40.424020+0000","last_peered":"2024-09-11T05:13:40.424020+0000","last_clean":"2024-09-11T05:13:40.424020+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:13:40.424020+0000","last_undegraded":"2024-09-11T05:13:40.424020+0000","last_fullsized":"2024-09-11T05:13:40.424020+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1129,"log_dups_size":0,"ondisk_log_size":1129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4624384,"num_objects":1129,"num_object_clones":0,"num_object_copies":2258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1129,"num_write_kb":4516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'830","reported_seq":891,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.427592+0000","last_change":"2024-09-11T05:13:21.517179+0000","last_active":"2024-09-11T05:13:38.427592+0000","last_peered":"2024-09-11T05:13:38.427592+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:13:38.427592+0000","last_undegraded":"2024-09-11T05:13:21.496359+0000","last_fullsized":"2024-09-11T05:13:38.427592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3380434,"num_objects":827,"num_object_clones":0,"num_object_copies":1654,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":830,"num_write_kb":3304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":827}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'848","reported_seq":907,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.427681+0000","last_change":"2024-09-11T05:13:21.514194+0000","last_active":"2024-09-11T05:13:38.427681+0000","last_peered":"2024-09-11T05:13:38.427681+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:13:21.512885+0000","last_became_peered":"2024-09-11T05:13:21.512885+0000","last_unstale":"2024-09-11T05:13:38.427681+0000","last_undegraded":"2024-09-11T05:13:21.492320+0000","last_fullsized":"2024-09-11T05:13:38.427681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3462354,"num_objects":847,"num_object_clones":0,"num_object_copies":1694,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":848,"num_write_kb":3383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":847}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'886","reported_seq":945,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.433725+0000","last_change":"2024-09-11T05:13:21.511966+0000","last_active":"2024-09-11T05:13:38.433725+0000","last_peered":"2024-09-11T05:13:38.433725+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:13:21.511332+0000","last_became_peered":"2024-09-11T05:13:21.511332+0000","last_unstale":"2024-09-11T05:13:38.433725+0000","last_undegraded":"2024-09-11T05:13:21.492627+0000","last_fullsized":"2024-09-11T05:13:38.433725+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3613906,"num_objects":884,"num_object_clones":0,"num_object_copies":1768,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":886,"num_write_kb":3532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":884}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'878","reported_seq":940,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-11T05:13:38.434041+0000","last_change":"2024-09-11T05:13:22.493892+0000","last_active":"2024-09-11T05:13:38.434041+0000","last_peered":"2024-09-11T05:13:38.434041+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:13:38.434041+0000","last_undegraded":"2024-09-11T05:13:38.434041+0000","last_fullsized":"2024-09-11T05:13:38.434041+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3592809,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":878,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":878}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.493688+0000","last_change":"2024-09-11T05:13:21.508556+0000","last_active":"2024-09-11T05:13:22.493688+0000","last_peered":"2024-09-11T05:13:22.493688+0000","last_clean":"2024-09-11T05:13:22.493688+0000","last_became_active":"2024-09-11T05:13:21.507784+0000","last_became_peered":"2024-09-11T05:13:21.507784+0000","last_unstale":"2024-09-11T05:13:22.493688+0000","last_undegraded":"2024-09-11T05:13:22.493688+0000","last_fullsized":"2024-09-11T05:13:22.493688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:10.209768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'1111","reported_seq":1174,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T05:13:40.427865+0000","last_change":"2024-09-11T05:13:25.436201+0000","last_active":"2024-09-11T05:13:40.427865+0000","last_peered":"2024-09-11T05:13:40.427865+0000","last_clean":"2024-09-11T05:12:43.565164+0000","last_became_active":"2024-09-11T05:13:22.496902+0000","last_became_peered":"2024-09-11T05:13:22.496902+0000","last_unstale":"2024-09-11T05:13:40.427865+0000","last_undegraded":"2024-09-11T05:13:22.500791+0000","last_fullsized":"2024-09-11T05:13:40.427865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4531410,"num_objects":1108,"num_object_clones":0,"num_object_copies":2216,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1111,"num_write_kb":4428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497168+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:13:22.497168+0000","last_peered":"2024-09-11T05:13:22.497168+0000","last_clean":"2024-09-11T05:13:22.497168+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:13:22.497168+0000","last_undegraded":"2024-09-11T05:13:22.497168+0000","last_fullsized":"2024-09-11T05:13:22.497168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'823","reported_seq":873,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.441395+0000","last_change":"2024-09-11T05:13:21.514035+0000","last_active":"2024-09-11T05:13:38.441395+0000","last_peered":"2024-09-11T05:13:38.441395+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:13:21.511727+0000","last_became_peered":"2024-09-11T05:13:21.511727+0000","last_unstale":"2024-09-11T05:13:38.441395+0000","last_undegraded":"2024-09-11T05:13:21.493049+0000","last_fullsized":"2024-09-11T05:13:38.441395+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3347666,"num_objects":819,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":823,"num_write_kb":3272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493699+0000","last_change":"2024-09-11T05:13:21.513836+0000","last_active":"2024-09-11T05:13:22.493699+0000","last_peered":"2024-09-11T05:13:22.493699+0000","last_clean":"2024-09-11T05:12:55.450586+0000","last_became_active":"2024-09-11T05:13:21.506029+0000","last_became_peered":"2024-09-11T05:13:21.506029+0000","last_unstale":"2024-09-11T05:13:22.493699+0000","last_undegraded":"2024-09-11T05:13:21.491804+0000","last_fullsized":"2024-09-11T05:13:22.493699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'1040","reported_seq":1092,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T05:13:40.434659+0000","last_change":"2024-09-11T05:13:33.069953+0000","last_active":"2024-09-11T05:13:40.434659+0000","last_peered":"2024-09-11T05:13:40.434659+0000","last_clean":"2024-09-11T05:12:42.358674+0000","last_became_active":"2024-09-11T05:13:22.495923+0000","last_became_peered":"2024-09-11T05:13:22.495923+0000","last_unstale":"2024-09-11T05:13:40.434659+0000","last_undegraded":"2024-09-11T05:13:22.497240+0000","last_fullsized":"2024-09-11T05:13:40.434659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4248786,"num_objects":1039,"num_object_clones":0,"num_object_copies":2078,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1040,"num_write_kb":4151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1039}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505097+0000","last_change":"2024-09-11T05:13:22.497695+0000","last_active":"2024-09-11T05:13:22.505097+0000","last_peered":"2024-09-11T05:13:22.505097+0000","last_clean":"2024-09-11T05:13:22.505097+0000","last_became_active":"2024-09-11T05:13:22.497288+0000","last_became_peered":"2024-09-11T05:13:22.497288+0000","last_unstale":"2024-09-11T05:13:22.505097+0000","last_undegraded":"2024-09-11T05:13:22.505097+0000","last_fullsized":"2024-09-11T05:13:22.505097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:56.694811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'1075","reported_seq":1127,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T05:13:40.411399+0000","last_change":"2024-09-11T05:13:33.073350+0000","last_active":"2024-09-11T05:13:40.411399+0000","last_peered":"2024-09-11T05:13:40.411399+0000","last_clean":"2024-09-11T05:12:42.979390+0000","last_became_active":"2024-09-11T05:13:22.496829+0000","last_became_peered":"2024-09-11T05:13:22.496829+0000","last_unstale":"2024-09-11T05:13:40.411399+0000","last_undegraded":"2024-09-11T05:13:22.500173+0000","last_fullsized":"2024-09-11T05:13:40.411399+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4388050,"num_objects":1073,"num_object_clones":0,"num_object_copies":2146,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1075,"num_write_kb":4287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1073}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'861","reported_seq":918,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.427519+0000","last_change":"2024-09-11T05:13:21.512469+0000","last_active":"2024-09-11T05:13:38.427519+0000","last_peered":"2024-09-11T05:13:38.427519+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:13:21.511812+0000","last_became_peered":"2024-09-11T05:13:21.511812+0000","last_unstale":"2024-09-11T05:13:38.427519+0000","last_undegraded":"2024-09-11T05:13:21.496652+0000","last_fullsized":"2024-09-11T05:13:38.427519+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3523177,"num_objects":861,"num_object_clones":0,"num_object_copies":1722,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":861,"num_write_kb":3441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":861}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.494954+0000","last_change":"2024-09-11T05:13:21.512849+0000","last_active":"2024-09-11T05:13:22.494954+0000","last_peered":"2024-09-11T05:13:22.494954+0000","last_clean":"2024-09-11T05:13:22.494954+0000","last_became_active":"2024-09-11T05:13:21.512089+0000","last_became_peered":"2024-09-11T05:13:21.512089+0000","last_unstale":"2024-09-11T05:13:22.494954+0000","last_undegraded":"2024-09-11T05:13:22.494954+0000","last_fullsized":"2024-09-11T05:13:22.494954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:09:25.087655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'856","reported_seq":915,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.440052+0000","last_change":"2024-09-11T05:13:21.516438+0000","last_active":"2024-09-11T05:13:38.440052+0000","last_peered":"2024-09-11T05:13:38.440052+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:13:21.515407+0000","last_became_peered":"2024-09-11T05:13:21.515407+0000","last_unstale":"2024-09-11T05:13:38.440052+0000","last_undegraded":"2024-09-11T05:13:21.496548+0000","last_fullsized":"2024-09-11T05:13:38.440052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3487547,"num_objects":854,"num_object_clones":0,"num_object_copies":1708,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":3408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":854}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497841+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:13:22.497841+0000","last_peered":"2024-09-11T05:13:22.497841+0000","last_clean":"2024-09-11T05:13:22.497841+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:13:22.497841+0000","last_undegraded":"2024-09-11T05:13:22.497841+0000","last_fullsized":"2024-09-11T05:13:22.497841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'856","reported_seq":914,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.428091+0000","last_change":"2024-09-11T05:13:21.516118+0000","last_active":"2024-09-11T05:13:38.428091+0000","last_peered":"2024-09-11T05:13:38.428091+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:13:21.515226+0000","last_became_peered":"2024-09-11T05:13:21.515226+0000","last_unstale":"2024-09-11T05:13:38.428091+0000","last_undegraded":"2024-09-11T05:13:21.494389+0000","last_fullsized":"2024-09-11T05:13:38.428091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3499218,"num_objects":856,"num_object_clones":0,"num_object_copies":1712,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":3418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":856}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495153+0000","last_change":"2024-09-11T05:13:21.511859+0000","last_active":"2024-09-11T05:13:22.495153+0000","last_peered":"2024-09-11T05:13:22.495153+0000","last_clean":"2024-09-11T05:13:22.495153+0000","last_became_active":"2024-09-11T05:13:21.511091+0000","last_became_peered":"2024-09-11T05:13:21.511091+0000","last_unstale":"2024-09-11T05:13:22.495153+0000","last_undegraded":"2024-09-11T05:13:22.495153+0000","last_fullsized":"2024-09-11T05:13:22.495153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:05.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505152+0000","last_change":"2024-09-11T05:13:22.498193+0000","last_active":"2024-09-11T05:13:22.505152+0000","last_peered":"2024-09-11T05:13:22.505152+0000","last_clean":"2024-09-11T05:13:22.505152+0000","last_became_active":"2024-09-11T05:13:22.498019+0000","last_became_peered":"2024-09-11T05:13:22.498019+0000","last_unstale":"2024-09-11T05:13:22.505152+0000","last_undegraded":"2024-09-11T05:13:22.505152+0000","last_fullsized":"2024-09-11T05:13:22.505152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:24.352250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'885","reported_seq":945,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:38.442359+0000","last_change":"2024-09-11T05:13:21.513883+0000","last_active":"2024-09-11T05:13:38.442359+0000","last_peered":"2024-09-11T05:13:38.442359+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:13:21.512960+0000","last_became_peered":"2024-09-11T05:13:21.512960+0000","last_unstale":"2024-09-11T05:13:38.442359+0000","last_undegraded":"2024-09-11T05:13:21.493149+0000","last_fullsized":"2024-09-11T05:13:38.442359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3590564,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":885,"num_write_kb":3511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":879}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":61580156,"num_objects":15058,"num_object_clones":0,"num_object_copies":30116,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":15058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15085,"num_write_kb":60166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":128204800,"data_stored":128072598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15085,"ondisk_log_size":15085,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986184,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138572,"kb_used_data":70780,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94233268,"statfs":{"total":96636764160,"available":96494866432,"internally_reserved":0,"allocated":72478720,"data_stored":72271144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":69398187},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215117,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111072,"kb_used_data":53072,"kb_used_omap":4,"kb_used_meta":57979,"kb_avail":94260768,"statfs":{"total":96636764160,"available":96523026432,"internally_reserved":0,"allocated":54345728,"data_stored":54099285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":59371179},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,7,4,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":55545856,"data_stored":55511066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72544256,"data_stored":72544256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:43.535 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-09-11T05:13:43.535 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-11T05:13:43.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-11T05:13:43.755 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:43.755 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:43.756 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_4", 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 1224592654, 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:43.757 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'3", 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'1", 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_6", 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:43.758 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 3101443614, 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'4", 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'2", 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:43.759 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "NotRecovering", 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:43.760 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:43.785 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-09-11T05:13:43.785 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-09-11T05:13:44.011 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:44.046 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-11T05:13:44.046 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-11T05:13:44.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 11 20 24128 24108 8.5583 6.03125 0.0109484 0.00912147 2024-09-11T05:13:44.249 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:44.249 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:44.249 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:44.249 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:44.249 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:44.249 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 1, 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 1, 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_7", 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:44.250 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 2358579165, 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'3", 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "NotRecovering", 2024-09-11T05:13:44.251 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:44.252 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:44.275 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-09-11T05:13:44.275 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-09-11T05:13:44.558 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T05:13:44.584 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-09-11T05:13:44.584 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-11T05:13:44.584 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-11T05:13:44.800 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:44.800 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:44.800 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:44.800 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:44.800 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:44.800 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_3", 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:44.801 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 943960171, 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'5", 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:44.802 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_1", 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 2885884443, 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'4", 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'1", 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:44.803 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:44.804 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:44.851 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-09-11T05:13:44.852 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-09-11T05:13:45.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 12 20 25572 25552 8.31517 5.64062 0.00951207 0.00938706 2024-09-11T05:13:45.124 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:45.153 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-11T05:13:45.153 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:45.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_9", 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 1217469130, 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:45.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'3", 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'1", 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_9", 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 170713514, 2024-09-11T05:13:45.385 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'2", 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:45.386 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:45.387 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:45.387 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:45.387 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:45.387 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:45.387 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:45.387 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:45.422 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-09-11T05:13:45.423 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-09-11T05:13:45.726 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:45.774 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-11T05:13:45.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:46.014 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_5", 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 2998549577, 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:46.015 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'4", 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'2", 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_2", 2024-09-11T05:13:46.016 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 4032093161, 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'3", 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'1", 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:46.017 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:46.018 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:46.047 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-09-11T05:13:46.047 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-09-11T05:13:46.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 13 20 27063 27043 8.12355 5.82422 0.0050426 0.00961072 2024-09-11T05:13:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:46.327 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovering 2024-09-11T05:13:46.327 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-11T05:13:46.623 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 1, 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 1, 2024-09-11T05:13:46.624 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_8", 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 2730327960, 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'1", 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:46.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:46.627 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:46.660 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-09-11T05:13:46.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-09-11T05:13:46.892 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T05:13:46.932 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-09-11T05:13:46.933 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-11T05:13:46.933 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-11T05:13:47.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 14 19 28551 28532 7.95875 5.81641 0.0230351 0.00980607 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:47.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_8", 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 1901787523, 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:47.161 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'5", 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'3", 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:47.162 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_1", 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 3122379363, 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'4", 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:47.163 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "NotRecovering", 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:47.164 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:47.165 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:47.165 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:47.184 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-09-11T05:13:47.185 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-09-11T05:13:47.409 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:47.453 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-09-11T05:13:47.453 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-11T05:13:47.453 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-11T05:13:47.458 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:13:47.457+0000 7f9b77f07640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs recovery_unfound (PG_DAMAGED) 2024-09-11T05:13:47.665 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:47.666 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_6", 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 1708673474, 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'5", 2024-09-11T05:13:47.667 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_9", 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 831402034, 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:47.668 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'6", 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:47.669 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:47.672 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:47.673 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:47.673 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:47.692 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-09-11T05:13:47.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-09-11T05:13:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:48.040 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-09-11T05:13:48.040 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-11T05:13:48.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-11T05:13:48.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 15 20 29930 29910 7.78702 5.38281 0.0121045 0.0100247 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:48.319 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_3", 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 459768081, 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'3", 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'1", 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_3", 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:48.321 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 480221105, 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'2", 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:48.322 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "NotRecovering", 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:48.323 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:48.356 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-09-11T05:13:48.356 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-09-11T05:13:48.643 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:48.683 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-09-11T05:13:48.683 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-11T05:13:48.683 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-11T05:13:48.891 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_5", 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 4248945696, 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'3", 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.894 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_5", 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 4212789968, 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:48.895 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'4", 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:48.896 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:48.897 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "NotRecovering", 2024-09-11T05:13:48.897 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:48.897 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:48.897 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:48.897 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:48.900 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:48.900 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:48.901 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:48.901 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:48.901 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:48.922 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-09-11T05:13:48.922 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-09-11T05:13:49.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 16 20 31321 31301 7.63993 5.43359 0.00663127 0.0102146 2024-09-11T05:13:49.201 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:49.231 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-11T05:13:49.231 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:49.531 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 1, 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 1, 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_7", 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 3761017527, 2024-09-11T05:13:49.532 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'1", 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:49.533 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:49.534 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:49.561 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-09-11T05:13:49.561 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-09-11T05:13:49.820 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T05:13:49.848 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-09-11T05:13:49.848 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-11T05:13:49.848 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-11T05:13:50.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 17 20 32663 32643 7.49771 5.24219 0.0144436 0.0104128 2024-09-11T05:13:50.166 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:50.166 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:50.166 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:50.166 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:50.166 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:50.166 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:50.166 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 3, 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 3, 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:50.167 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_8", 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 1861424966, 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.168 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'5", 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_6", 2024-09-11T05:13:50.169 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 3910131350, 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'4", 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:50.170 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_4", 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:50.171 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 2787839990, 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'3", 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:50.172 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:50.174 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:50.194 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-09-11T05:13:50.194 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-09-11T05:13:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-11T05:13:50.557 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-09-11T05:13:50.557 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-11T05:13:50.557 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-11T05:13:50.803 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:50.803 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:50.803 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:50.803 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:50.803 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:50.803 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 2, 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 2, 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_1", 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:50.804 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 26281365, 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'1", 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:50.805 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "new_2", 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 3196340981, 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'2", 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:50.806 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:50.807 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:50.840 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-09-11T05:13:50.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-09-11T05:13:51.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 18 20 33939 33919 7.35809 4.98438 0.0279282 0.0106096 2024-09-11T05:13:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T05:13:51.145 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-09-11T05:13:51.145 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-09-11T05:13:51.145 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-11T05:13:51.145 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "offset": { 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "", 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": 0, 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 0, 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": -9223372036854775808, 2024-09-11T05:13:51.430 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "num_missing": 4, 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "num_unfound": 3, 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": [ 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_4", 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 873891716, 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:51.431 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'8", 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existing_7", 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:51.432 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 3996155460, 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'9", 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "20'6", 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": { 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "oid": "existed_2", 2024-09-11T05:13:51.433 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "", 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "snapid": -2, 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "hash": 1767667508, 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 0, 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 2, 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "namespace": "" 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "need": "22'7", 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "have": "0'0", 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": "none", 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: "locations": [] 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:51.434 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "Active", 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: "available_might_have_unfound": true, 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: "might_have_unfound": [ 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "1", 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "osd is down" 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout: "more": false 2024-09-11T05:13:51.435 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:13:51.455 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-09-11T05:13:51.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-09-11T05:13:51.669 INFO:teuthology.orchestra.run.smithi133.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-11T05:13:51.704 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-11T05:13:51.925 INFO:teuthology.orchestra.run.smithi133.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-11T05:13:51.949 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-11T05:13:52.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 19 20 35296 35276 7.24982 5.30078 0.00807879 0.0107694 2024-09-11T05:13:52.200 INFO:teuthology.orchestra.run.smithi133.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-11T05:13:52.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T05:13:52.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T05:13:52.499 INFO:teuthology.orchestra.run.smithi133.stdout:133143986187 2024-09-11T05:13:52.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:13:52.531 INFO:teuthology.orchestra.run.smithi133.stdout:103079215120 2024-09-11T05:13:52.531 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:13:53.031 INFO:teuthology.orchestra.run.smithi133.stdout:133143986186 2024-09-11T05:13:53.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986187 got 133143986186 for osd.2 2024-09-11T05:13:53.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:13:53.072441+0000 min lat: 0.00201616 max lat: 0.0637779 avg lat: 0.0109549 2024-09-11T05:13:53.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:13:53.071 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 20 20 36512 36492 7.12484 4.75 0.0604447 0.0109549 2024-09-11T05:13:53.300 INFO:teuthology.orchestra.run.smithi133.stdout:103079215119 2024-09-11T05:13:53.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215120 got 103079215119 for osd.0 2024-09-11T05:13:54.060 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T05:13:54.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 21 20 37951 37931 7.05323 5.62109 0.0185431 0.0110692 2024-09-11T05:13:54.334 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T05:13:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:133143986187 2024-09-11T05:13:54.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986187 got 133143986187 for osd.2 2024-09-11T05:13:54.723 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:54.879 INFO:teuthology.orchestra.run.smithi133.stdout:103079215120 2024-09-11T05:13:54.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 103079215120 got 103079215120 for osd.0 2024-09-11T05:13:54.909 DEBUG:teuthology.parallel:result is None 2024-09-11T05:13:54.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-11T05:13:54.909 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:55.070 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 22 20 39175 39155 6.94996 4.78125 0.0110136 0.0112346 2024-09-11T05:13:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:55.439 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:55.489 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-09-11T05:13:55.367743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149872094,"num_objects":36608,"num_object_clones":0,"num_object_copies":73216,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27454,"num_write_kb":109696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36672,"ondisk_log_size":36672,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":487372,"kb_used_data":289384,"kb_used_omap":8,"kb_used_meta":197943,"kb_avail":188256308,"statfs":{"total":193273528320,"available":192774459392,"internally_reserved":0,"allocated":296329216,"data_stored":295889113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":202693974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,18,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88291919,"num_objects":21548,"num_object_clones":0,"num_object_copies":43096,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":21548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12367,"num_write_kb":49528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002844"},"pg_stats":[{"pgid":"2.f","version":"36'2372","reported_seq":2425,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:53.482443+0000","last_change":"2024-09-11T05:13:21.515668+0000","last_active":"2024-09-11T05:13:53.482443+0000","last_peered":"2024-09-11T05:13:53.482443+0000","last_clean":"2024-09-11T05:13:53.482443+0000","last_became_active":"2024-09-11T05:13:21.514919+0000","last_became_peered":"2024-09-11T05:13:21.514919+0000","last_unstale":"2024-09-11T05:13:53.482443+0000","last_undegraded":"2024-09-11T05:13:53.482443+0000","last_fullsized":"2024-09-11T05:13:53.482443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.714157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9715712,"num_objects":2372,"num_object_clones":0,"num_object_copies":4744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2257","reported_seq":2340,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.432277+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:13:52.432277+0000","last_peered":"2024-09-11T05:13:52.432277+0000","last_clean":"2024-09-11T05:13:52.432277+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:13:52.432277+0000","last_undegraded":"2024-09-11T05:13:52.432277+0000","last_fullsized":"2024-09-11T05:13:52.432277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":2257,"log_dups_size":0,"ondisk_log_size":2257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9220096,"num_objects":2251,"num_object_clones":0,"num_object_copies":4502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":655,"num_write_kb":2620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2170","reported_seq":2268,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.415742+0000","last_change":"2024-09-11T05:13:45.985545+0000","last_active":"2024-09-11T05:13:52.415742+0000","last_peered":"2024-09-11T05:13:52.415742+0000","last_clean":"2024-09-11T05:13:52.415742+0000","last_became_active":"2024-09-11T05:13:22.497399+0000","last_became_peered":"2024-09-11T05:13:22.497399+0000","last_unstale":"2024-09-11T05:13:52.415742+0000","last_undegraded":"2024-09-11T05:13:52.415742+0000","last_fullsized":"2024-09-11T05:13:52.415742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":2170,"log_dups_size":0,"ondisk_log_size":2170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:07.277685+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8871936,"num_objects":2166,"num_object_clones":0,"num_object_copies":4332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":573,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2279","reported_seq":2331,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.440542+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:13:52.440542+0000","last_peered":"2024-09-11T05:13:52.440542+0000","last_clean":"2024-09-11T05:13:52.440542+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:13:52.440542+0000","last_undegraded":"2024-09-11T05:13:52.440542+0000","last_fullsized":"2024-09-11T05:13:52.440542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2279,"log_dups_size":0,"ondisk_log_size":2279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9334784,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2279,"num_write_kb":9116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2355","reported_seq":2414,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480517+0000","last_change":"2024-09-11T05:13:21.517179+0000","last_active":"2024-09-11T05:13:53.480517+0000","last_peered":"2024-09-11T05:13:53.480517+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:13:53.480517+0000","last_undegraded":"2024-09-11T05:13:21.496359+0000","last_fullsized":"2024-09-11T05:13:53.480517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2355,"log_dups_size":0,"ondisk_log_size":2355,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9618642,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2353,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2412","reported_seq":2470,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.456273+0000","last_change":"2024-09-11T05:13:21.514194+0000","last_active":"2024-09-11T05:13:53.456273+0000","last_peered":"2024-09-11T05:13:53.456273+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:13:21.512885+0000","last_became_peered":"2024-09-11T05:13:21.512885+0000","last_unstale":"2024-09-11T05:13:53.456273+0000","last_undegraded":"2024-09-11T05:13:21.492320+0000","last_fullsized":"2024-09-11T05:13:53.456273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2412,"log_dups_size":0,"ondisk_log_size":2412,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9860306,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2410,"num_write_kb":9631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2363","reported_seq":2421,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.456185+0000","last_change":"2024-09-11T05:13:21.511966+0000","last_active":"2024-09-11T05:13:53.456185+0000","last_peered":"2024-09-11T05:13:53.456185+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:13:21.511332+0000","last_became_peered":"2024-09-11T05:13:21.511332+0000","last_unstale":"2024-09-11T05:13:53.456185+0000","last_undegraded":"2024-09-11T05:13:21.492627+0000","last_fullsized":"2024-09-11T05:13:53.456185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9655506,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":9432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2312","reported_seq":2375,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-11T05:13:53.478260+0000","last_change":"2024-09-11T05:13:22.493892+0000","last_active":"2024-09-11T05:13:53.478260+0000","last_peered":"2024-09-11T05:13:53.478260+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:13:53.478260+0000","last_undegraded":"2024-09-11T05:13:53.478260+0000","last_fullsized":"2024-09-11T05:13:53.478260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462377,"num_objects":2311,"num_object_clones":0,"num_object_copies":4622,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2311,"num_write_kb":9241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.493688+0000","last_change":"2024-09-11T05:13:21.508556+0000","last_active":"2024-09-11T05:13:22.493688+0000","last_peered":"2024-09-11T05:13:22.493688+0000","last_clean":"2024-09-11T05:13:22.493688+0000","last_became_active":"2024-09-11T05:13:21.507784+0000","last_became_peered":"2024-09-11T05:13:21.507784+0000","last_unstale":"2024-09-11T05:13:22.493688+0000","last_undegraded":"2024-09-11T05:13:22.493688+0000","last_fullsized":"2024-09-11T05:13:22.493688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:10.209768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2256","reported_seq":2362,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.456494+0000","last_change":"2024-09-11T05:13:49.372903+0000","last_active":"2024-09-11T05:13:52.456494+0000","last_peered":"2024-09-11T05:13:52.456494+0000","last_clean":"2024-09-11T05:13:52.456494+0000","last_became_active":"2024-09-11T05:13:22.496902+0000","last_became_peered":"2024-09-11T05:13:22.496902+0000","last_unstale":"2024-09-11T05:13:52.456494+0000","last_undegraded":"2024-09-11T05:13:52.456494+0000","last_fullsized":"2024-09-11T05:13:52.456494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":2256,"log_dups_size":0,"ondisk_log_size":2256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:47.496372+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9211904,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":1296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497168+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:13:22.497168+0000","last_peered":"2024-09-11T05:13:22.497168+0000","last_clean":"2024-09-11T05:13:22.497168+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:13:22.497168+0000","last_undegraded":"2024-09-11T05:13:22.497168+0000","last_fullsized":"2024-09-11T05:13:22.497168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2240","reported_seq":2291,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.472695+0000","last_change":"2024-09-11T05:13:21.514035+0000","last_active":"2024-09-11T05:13:53.472695+0000","last_peered":"2024-09-11T05:13:53.472695+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:13:21.511727+0000","last_became_peered":"2024-09-11T05:13:21.511727+0000","last_unstale":"2024-09-11T05:13:53.472695+0000","last_undegraded":"2024-09-11T05:13:21.493049+0000","last_fullsized":"2024-09-11T05:13:53.472695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9143506,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2238,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493699+0000","last_change":"2024-09-11T05:13:21.513836+0000","last_active":"2024-09-11T05:13:22.493699+0000","last_peered":"2024-09-11T05:13:22.493699+0000","last_clean":"2024-09-11T05:12:55.450586+0000","last_became_active":"2024-09-11T05:13:21.506029+0000","last_became_peered":"2024-09-11T05:13:21.506029+0000","last_unstale":"2024-09-11T05:13:22.493699+0000","last_undegraded":"2024-09-11T05:13:21.491804+0000","last_fullsized":"2024-09-11T05:13:22.493699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2193","reported_seq":2285,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.432064+0000","last_change":"2024-09-11T05:13:49.998432+0000","last_active":"2024-09-11T05:13:52.432064+0000","last_peered":"2024-09-11T05:13:52.432064+0000","last_clean":"2024-09-11T05:13:52.432064+0000","last_became_active":"2024-09-11T05:13:22.495923+0000","last_became_peered":"2024-09-11T05:13:22.495923+0000","last_unstale":"2024-09-11T05:13:52.432064+0000","last_undegraded":"2024-09-11T05:13:52.432064+0000","last_fullsized":"2024-09-11T05:13:52.432064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":2193,"log_dups_size":0,"ondisk_log_size":2193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:19.484483+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8962048,"num_objects":2188,"num_object_clones":0,"num_object_copies":4376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505097+0000","last_change":"2024-09-11T05:13:22.497695+0000","last_active":"2024-09-11T05:13:22.505097+0000","last_peered":"2024-09-11T05:13:22.505097+0000","last_clean":"2024-09-11T05:13:22.505097+0000","last_became_active":"2024-09-11T05:13:22.497288+0000","last_became_peered":"2024-09-11T05:13:22.497288+0000","last_unstale":"2024-09-11T05:13:22.505097+0000","last_undegraded":"2024-09-11T05:13:22.505097+0000","last_fullsized":"2024-09-11T05:13:22.505097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:56.694811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2234","reported_seq":2327,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.443057+0000","last_change":"2024-09-11T05:13:51.225829+0000","last_active":"2024-09-11T05:13:52.443057+0000","last_peered":"2024-09-11T05:13:52.443057+0000","last_clean":"2024-09-11T05:13:52.443057+0000","last_became_active":"2024-09-11T05:13:22.496829+0000","last_became_peered":"2024-09-11T05:13:22.496829+0000","last_unstale":"2024-09-11T05:13:52.443057+0000","last_undegraded":"2024-09-11T05:13:52.443057+0000","last_fullsized":"2024-09-11T05:13:52.443057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:27.126916+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9125888,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":137,"num_write_kb":548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2348","reported_seq":2406,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.455822+0000","last_change":"2024-09-11T05:13:21.512469+0000","last_active":"2024-09-11T05:13:53.455822+0000","last_peered":"2024-09-11T05:13:53.455822+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:13:21.511812+0000","last_became_peered":"2024-09-11T05:13:21.511812+0000","last_unstale":"2024-09-11T05:13:53.455822+0000","last_undegraded":"2024-09-11T05:13:21.496652+0000","last_fullsized":"2024-09-11T05:13:53.455822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2348,"log_dups_size":0,"ondisk_log_size":2348,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9609833,"num_objects":2347,"num_object_clones":0,"num_object_copies":4694,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2347,"num_write_kb":9385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2347}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.494954+0000","last_change":"2024-09-11T05:13:21.512849+0000","last_active":"2024-09-11T05:13:22.494954+0000","last_peered":"2024-09-11T05:13:22.494954+0000","last_clean":"2024-09-11T05:13:22.494954+0000","last_became_active":"2024-09-11T05:13:21.512089+0000","last_became_peered":"2024-09-11T05:13:21.512089+0000","last_unstale":"2024-09-11T05:13:22.494954+0000","last_undegraded":"2024-09-11T05:13:22.494954+0000","last_fullsized":"2024-09-11T05:13:22.494954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:09:25.087655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2261","reported_seq":2318,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480418+0000","last_change":"2024-09-11T05:13:21.516438+0000","last_active":"2024-09-11T05:13:53.480418+0000","last_peered":"2024-09-11T05:13:53.480418+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:13:21.515407+0000","last_became_peered":"2024-09-11T05:13:21.515407+0000","last_unstale":"2024-09-11T05:13:53.480418+0000","last_undegraded":"2024-09-11T05:13:21.496548+0000","last_fullsized":"2024-09-11T05:13:53.480418+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9230139,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2258,"num_write_kb":9016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2256}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497841+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:13:22.497841+0000","last_peered":"2024-09-11T05:13:22.497841+0000","last_clean":"2024-09-11T05:13:22.497841+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:13:22.497841+0000","last_undegraded":"2024-09-11T05:13:22.497841+0000","last_fullsized":"2024-09-11T05:13:22.497841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2310","reported_seq":2367,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.472969+0000","last_change":"2024-09-11T05:13:21.516118+0000","last_active":"2024-09-11T05:13:53.472969+0000","last_peered":"2024-09-11T05:13:53.472969+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:13:21.515226+0000","last_became_peered":"2024-09-11T05:13:21.515226+0000","last_unstale":"2024-09-11T05:13:53.472969+0000","last_undegraded":"2024-09-11T05:13:21.494389+0000","last_fullsized":"2024-09-11T05:13:53.472969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9446610,"num_objects":2308,"num_object_clones":0,"num_object_copies":4616,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2308,"num_write_kb":9226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2308}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495153+0000","last_change":"2024-09-11T05:13:21.511859+0000","last_active":"2024-09-11T05:13:22.495153+0000","last_peered":"2024-09-11T05:13:22.495153+0000","last_clean":"2024-09-11T05:13:22.495153+0000","last_became_active":"2024-09-11T05:13:21.511091+0000","last_became_peered":"2024-09-11T05:13:21.511091+0000","last_unstale":"2024-09-11T05:13:22.495153+0000","last_undegraded":"2024-09-11T05:13:22.495153+0000","last_fullsized":"2024-09-11T05:13:22.495153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:05.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505152+0000","last_change":"2024-09-11T05:13:22.498193+0000","last_active":"2024-09-11T05:13:22.505152+0000","last_peered":"2024-09-11T05:13:22.505152+0000","last_clean":"2024-09-11T05:13:22.505152+0000","last_became_active":"2024-09-11T05:13:22.498019+0000","last_became_peered":"2024-09-11T05:13:22.498019+0000","last_unstale":"2024-09-11T05:13:22.505152+0000","last_undegraded":"2024-09-11T05:13:22.505152+0000","last_fullsized":"2024-09-11T05:13:22.505152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:24.352250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2307","reported_seq":2364,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480656+0000","last_change":"2024-09-11T05:13:21.513883+0000","last_active":"2024-09-11T05:13:53.480656+0000","last_peered":"2024-09-11T05:13:53.480656+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:13:21.512960+0000","last_became_peered":"2024-09-11T05:13:21.512960+0000","last_unstale":"2024-09-11T05:13:53.480656+0000","last_undegraded":"2024-09-11T05:13:21.493149+0000","last_fullsized":"2024-09-11T05:13:53.480656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2307,"log_dups_size":0,"ondisk_log_size":2307,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9402788,"num_objects":2299,"num_object_clones":0,"num_object_copies":4598,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2304,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149872075,"num_objects":36606,"num_object_clones":0,"num_object_copies":73212,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27452,"num_write_kb":109694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298520576,"data_stored":298402290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36669,"ondisk_log_size":36669,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232376,"kb_used_data":142376,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94139464,"statfs":{"total":96636764160,"available":96398811136,"internally_reserved":0,"allocated":145793024,"data_stored":145585448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":92139179},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215121,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254996,"kb_used_data":147008,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94116844,"statfs":{"total":96636764160,"available":96375648256,"internally_reserved":0,"allocated":150536192,"data_stored":150303665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110554795},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,18,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151957504,"data_stored":151936630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146448384,"data_stored":146448384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:55.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:56.074 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 23 20 40445 40425 6.86245 4.96094 0.00942605 0.0113748 2024-09-11T05:13:56.155 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:56.155 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:56.204 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-09-11T05:13:55.367743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149872094,"num_objects":36608,"num_object_clones":0,"num_object_copies":73216,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27454,"num_write_kb":109696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36672,"ondisk_log_size":36672,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":487372,"kb_used_data":289384,"kb_used_omap":8,"kb_used_meta":197943,"kb_avail":188256308,"statfs":{"total":193273528320,"available":192774459392,"internally_reserved":0,"allocated":296329216,"data_stored":295889113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":202693974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,18,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88291919,"num_objects":21548,"num_object_clones":0,"num_object_copies":43096,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":21548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12367,"num_write_kb":49528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002844"},"pg_stats":[{"pgid":"2.f","version":"36'2372","reported_seq":2425,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:53.482443+0000","last_change":"2024-09-11T05:13:21.515668+0000","last_active":"2024-09-11T05:13:53.482443+0000","last_peered":"2024-09-11T05:13:53.482443+0000","last_clean":"2024-09-11T05:13:53.482443+0000","last_became_active":"2024-09-11T05:13:21.514919+0000","last_became_peered":"2024-09-11T05:13:21.514919+0000","last_unstale":"2024-09-11T05:13:53.482443+0000","last_undegraded":"2024-09-11T05:13:53.482443+0000","last_fullsized":"2024-09-11T05:13:53.482443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.714157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9715712,"num_objects":2372,"num_object_clones":0,"num_object_copies":4744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2257","reported_seq":2340,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.432277+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:13:52.432277+0000","last_peered":"2024-09-11T05:13:52.432277+0000","last_clean":"2024-09-11T05:13:52.432277+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:13:52.432277+0000","last_undegraded":"2024-09-11T05:13:52.432277+0000","last_fullsized":"2024-09-11T05:13:52.432277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":2257,"log_dups_size":0,"ondisk_log_size":2257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9220096,"num_objects":2251,"num_object_clones":0,"num_object_copies":4502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":655,"num_write_kb":2620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2170","reported_seq":2268,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.415742+0000","last_change":"2024-09-11T05:13:45.985545+0000","last_active":"2024-09-11T05:13:52.415742+0000","last_peered":"2024-09-11T05:13:52.415742+0000","last_clean":"2024-09-11T05:13:52.415742+0000","last_became_active":"2024-09-11T05:13:22.497399+0000","last_became_peered":"2024-09-11T05:13:22.497399+0000","last_unstale":"2024-09-11T05:13:52.415742+0000","last_undegraded":"2024-09-11T05:13:52.415742+0000","last_fullsized":"2024-09-11T05:13:52.415742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":2170,"log_dups_size":0,"ondisk_log_size":2170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:07.277685+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8871936,"num_objects":2166,"num_object_clones":0,"num_object_copies":4332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":573,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2279","reported_seq":2331,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.440542+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:13:52.440542+0000","last_peered":"2024-09-11T05:13:52.440542+0000","last_clean":"2024-09-11T05:13:52.440542+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:13:52.440542+0000","last_undegraded":"2024-09-11T05:13:52.440542+0000","last_fullsized":"2024-09-11T05:13:52.440542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2279,"log_dups_size":0,"ondisk_log_size":2279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9334784,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2279,"num_write_kb":9116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2355","reported_seq":2414,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480517+0000","last_change":"2024-09-11T05:13:21.517179+0000","last_active":"2024-09-11T05:13:53.480517+0000","last_peered":"2024-09-11T05:13:53.480517+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:13:53.480517+0000","last_undegraded":"2024-09-11T05:13:21.496359+0000","last_fullsized":"2024-09-11T05:13:53.480517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2355,"log_dups_size":0,"ondisk_log_size":2355,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9618642,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2353,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2412","reported_seq":2470,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.456273+0000","last_change":"2024-09-11T05:13:21.514194+0000","last_active":"2024-09-11T05:13:53.456273+0000","last_peered":"2024-09-11T05:13:53.456273+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:13:21.512885+0000","last_became_peered":"2024-09-11T05:13:21.512885+0000","last_unstale":"2024-09-11T05:13:53.456273+0000","last_undegraded":"2024-09-11T05:13:21.492320+0000","last_fullsized":"2024-09-11T05:13:53.456273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2412,"log_dups_size":0,"ondisk_log_size":2412,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9860306,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2410,"num_write_kb":9631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2363","reported_seq":2421,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.456185+0000","last_change":"2024-09-11T05:13:21.511966+0000","last_active":"2024-09-11T05:13:53.456185+0000","last_peered":"2024-09-11T05:13:53.456185+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:13:21.511332+0000","last_became_peered":"2024-09-11T05:13:21.511332+0000","last_unstale":"2024-09-11T05:13:53.456185+0000","last_undegraded":"2024-09-11T05:13:21.492627+0000","last_fullsized":"2024-09-11T05:13:53.456185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9655506,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":9432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2312","reported_seq":2375,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-11T05:13:53.478260+0000","last_change":"2024-09-11T05:13:22.493892+0000","last_active":"2024-09-11T05:13:53.478260+0000","last_peered":"2024-09-11T05:13:53.478260+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:13:53.478260+0000","last_undegraded":"2024-09-11T05:13:53.478260+0000","last_fullsized":"2024-09-11T05:13:53.478260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462377,"num_objects":2311,"num_object_clones":0,"num_object_copies":4622,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2311,"num_write_kb":9241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.493688+0000","last_change":"2024-09-11T05:13:21.508556+0000","last_active":"2024-09-11T05:13:22.493688+0000","last_peered":"2024-09-11T05:13:22.493688+0000","last_clean":"2024-09-11T05:13:22.493688+0000","last_became_active":"2024-09-11T05:13:21.507784+0000","last_became_peered":"2024-09-11T05:13:21.507784+0000","last_unstale":"2024-09-11T05:13:22.493688+0000","last_undegraded":"2024-09-11T05:13:22.493688+0000","last_fullsized":"2024-09-11T05:13:22.493688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:10.209768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2256","reported_seq":2362,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.456494+0000","last_change":"2024-09-11T05:13:49.372903+0000","last_active":"2024-09-11T05:13:52.456494+0000","last_peered":"2024-09-11T05:13:52.456494+0000","last_clean":"2024-09-11T05:13:52.456494+0000","last_became_active":"2024-09-11T05:13:22.496902+0000","last_became_peered":"2024-09-11T05:13:22.496902+0000","last_unstale":"2024-09-11T05:13:52.456494+0000","last_undegraded":"2024-09-11T05:13:52.456494+0000","last_fullsized":"2024-09-11T05:13:52.456494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":2256,"log_dups_size":0,"ondisk_log_size":2256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:47.496372+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9211904,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":1296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497168+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:13:22.497168+0000","last_peered":"2024-09-11T05:13:22.497168+0000","last_clean":"2024-09-11T05:13:22.497168+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:13:22.497168+0000","last_undegraded":"2024-09-11T05:13:22.497168+0000","last_fullsized":"2024-09-11T05:13:22.497168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2240","reported_seq":2291,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.472695+0000","last_change":"2024-09-11T05:13:21.514035+0000","last_active":"2024-09-11T05:13:53.472695+0000","last_peered":"2024-09-11T05:13:53.472695+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:13:21.511727+0000","last_became_peered":"2024-09-11T05:13:21.511727+0000","last_unstale":"2024-09-11T05:13:53.472695+0000","last_undegraded":"2024-09-11T05:13:21.493049+0000","last_fullsized":"2024-09-11T05:13:53.472695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9143506,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2238,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493699+0000","last_change":"2024-09-11T05:13:21.513836+0000","last_active":"2024-09-11T05:13:22.493699+0000","last_peered":"2024-09-11T05:13:22.493699+0000","last_clean":"2024-09-11T05:12:55.450586+0000","last_became_active":"2024-09-11T05:13:21.506029+0000","last_became_peered":"2024-09-11T05:13:21.506029+0000","last_unstale":"2024-09-11T05:13:22.493699+0000","last_undegraded":"2024-09-11T05:13:21.491804+0000","last_fullsized":"2024-09-11T05:13:22.493699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2193","reported_seq":2285,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.432064+0000","last_change":"2024-09-11T05:13:49.998432+0000","last_active":"2024-09-11T05:13:52.432064+0000","last_peered":"2024-09-11T05:13:52.432064+0000","last_clean":"2024-09-11T05:13:52.432064+0000","last_became_active":"2024-09-11T05:13:22.495923+0000","last_became_peered":"2024-09-11T05:13:22.495923+0000","last_unstale":"2024-09-11T05:13:52.432064+0000","last_undegraded":"2024-09-11T05:13:52.432064+0000","last_fullsized":"2024-09-11T05:13:52.432064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":2193,"log_dups_size":0,"ondisk_log_size":2193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:19.484483+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8962048,"num_objects":2188,"num_object_clones":0,"num_object_copies":4376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505097+0000","last_change":"2024-09-11T05:13:22.497695+0000","last_active":"2024-09-11T05:13:22.505097+0000","last_peered":"2024-09-11T05:13:22.505097+0000","last_clean":"2024-09-11T05:13:22.505097+0000","last_became_active":"2024-09-11T05:13:22.497288+0000","last_became_peered":"2024-09-11T05:13:22.497288+0000","last_unstale":"2024-09-11T05:13:22.505097+0000","last_undegraded":"2024-09-11T05:13:22.505097+0000","last_fullsized":"2024-09-11T05:13:22.505097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:56.694811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2234","reported_seq":2327,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.443057+0000","last_change":"2024-09-11T05:13:51.225829+0000","last_active":"2024-09-11T05:13:52.443057+0000","last_peered":"2024-09-11T05:13:52.443057+0000","last_clean":"2024-09-11T05:13:52.443057+0000","last_became_active":"2024-09-11T05:13:22.496829+0000","last_became_peered":"2024-09-11T05:13:22.496829+0000","last_unstale":"2024-09-11T05:13:52.443057+0000","last_undegraded":"2024-09-11T05:13:52.443057+0000","last_fullsized":"2024-09-11T05:13:52.443057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:27.126916+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9125888,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":137,"num_write_kb":548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2348","reported_seq":2406,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.455822+0000","last_change":"2024-09-11T05:13:21.512469+0000","last_active":"2024-09-11T05:13:53.455822+0000","last_peered":"2024-09-11T05:13:53.455822+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:13:21.511812+0000","last_became_peered":"2024-09-11T05:13:21.511812+0000","last_unstale":"2024-09-11T05:13:53.455822+0000","last_undegraded":"2024-09-11T05:13:21.496652+0000","last_fullsized":"2024-09-11T05:13:53.455822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2348,"log_dups_size":0,"ondisk_log_size":2348,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9609833,"num_objects":2347,"num_object_clones":0,"num_object_copies":4694,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2347,"num_write_kb":9385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2347}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.494954+0000","last_change":"2024-09-11T05:13:21.512849+0000","last_active":"2024-09-11T05:13:22.494954+0000","last_peered":"2024-09-11T05:13:22.494954+0000","last_clean":"2024-09-11T05:13:22.494954+0000","last_became_active":"2024-09-11T05:13:21.512089+0000","last_became_peered":"2024-09-11T05:13:21.512089+0000","last_unstale":"2024-09-11T05:13:22.494954+0000","last_undegraded":"2024-09-11T05:13:22.494954+0000","last_fullsized":"2024-09-11T05:13:22.494954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:09:25.087655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2261","reported_seq":2318,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480418+0000","last_change":"2024-09-11T05:13:21.516438+0000","last_active":"2024-09-11T05:13:53.480418+0000","last_peered":"2024-09-11T05:13:53.480418+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:13:21.515407+0000","last_became_peered":"2024-09-11T05:13:21.515407+0000","last_unstale":"2024-09-11T05:13:53.480418+0000","last_undegraded":"2024-09-11T05:13:21.496548+0000","last_fullsized":"2024-09-11T05:13:53.480418+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9230139,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2258,"num_write_kb":9016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2256}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497841+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:13:22.497841+0000","last_peered":"2024-09-11T05:13:22.497841+0000","last_clean":"2024-09-11T05:13:22.497841+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:13:22.497841+0000","last_undegraded":"2024-09-11T05:13:22.497841+0000","last_fullsized":"2024-09-11T05:13:22.497841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2310","reported_seq":2367,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.472969+0000","last_change":"2024-09-11T05:13:21.516118+0000","last_active":"2024-09-11T05:13:53.472969+0000","last_peered":"2024-09-11T05:13:53.472969+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:13:21.515226+0000","last_became_peered":"2024-09-11T05:13:21.515226+0000","last_unstale":"2024-09-11T05:13:53.472969+0000","last_undegraded":"2024-09-11T05:13:21.494389+0000","last_fullsized":"2024-09-11T05:13:53.472969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9446610,"num_objects":2308,"num_object_clones":0,"num_object_copies":4616,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2308,"num_write_kb":9226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2308}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495153+0000","last_change":"2024-09-11T05:13:21.511859+0000","last_active":"2024-09-11T05:13:22.495153+0000","last_peered":"2024-09-11T05:13:22.495153+0000","last_clean":"2024-09-11T05:13:22.495153+0000","last_became_active":"2024-09-11T05:13:21.511091+0000","last_became_peered":"2024-09-11T05:13:21.511091+0000","last_unstale":"2024-09-11T05:13:22.495153+0000","last_undegraded":"2024-09-11T05:13:22.495153+0000","last_fullsized":"2024-09-11T05:13:22.495153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:05.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505152+0000","last_change":"2024-09-11T05:13:22.498193+0000","last_active":"2024-09-11T05:13:22.505152+0000","last_peered":"2024-09-11T05:13:22.505152+0000","last_clean":"2024-09-11T05:13:22.505152+0000","last_became_active":"2024-09-11T05:13:22.498019+0000","last_became_peered":"2024-09-11T05:13:22.498019+0000","last_unstale":"2024-09-11T05:13:22.505152+0000","last_undegraded":"2024-09-11T05:13:22.505152+0000","last_fullsized":"2024-09-11T05:13:22.505152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:24.352250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2307","reported_seq":2364,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480656+0000","last_change":"2024-09-11T05:13:21.513883+0000","last_active":"2024-09-11T05:13:53.480656+0000","last_peered":"2024-09-11T05:13:53.480656+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:13:21.512960+0000","last_became_peered":"2024-09-11T05:13:21.512960+0000","last_unstale":"2024-09-11T05:13:53.480656+0000","last_undegraded":"2024-09-11T05:13:21.493149+0000","last_fullsized":"2024-09-11T05:13:53.480656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2307,"log_dups_size":0,"ondisk_log_size":2307,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9402788,"num_objects":2299,"num_object_clones":0,"num_object_copies":4598,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2304,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149872075,"num_objects":36606,"num_object_clones":0,"num_object_copies":73212,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27452,"num_write_kb":109694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298520576,"data_stored":298402290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36669,"ondisk_log_size":36669,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232376,"kb_used_data":142376,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94139464,"statfs":{"total":96636764160,"available":96398811136,"internally_reserved":0,"allocated":145793024,"data_stored":145585448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":92139179},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215121,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254996,"kb_used_data":147008,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94116844,"statfs":{"total":96636764160,"available":96375648256,"internally_reserved":0,"allocated":150536192,"data_stored":150303665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110554795},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,18,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151957504,"data_stored":151936630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146448384,"data_stored":146448384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:56.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:13:56.676 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:13:56.677 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:13:56.725 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-09-11T05:13:55.367743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149872094,"num_objects":36608,"num_object_clones":0,"num_object_copies":73216,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27454,"num_write_kb":109696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36672,"ondisk_log_size":36672,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":487372,"kb_used_data":289384,"kb_used_omap":8,"kb_used_meta":197943,"kb_avail":188256308,"statfs":{"total":193273528320,"available":192774459392,"internally_reserved":0,"allocated":296329216,"data_stored":295889113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":202693974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,18,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88291919,"num_objects":21548,"num_object_clones":0,"num_object_copies":43096,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":21548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12367,"num_write_kb":49528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002844"},"pg_stats":[{"pgid":"2.f","version":"36'2372","reported_seq":2425,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:53.482443+0000","last_change":"2024-09-11T05:13:21.515668+0000","last_active":"2024-09-11T05:13:53.482443+0000","last_peered":"2024-09-11T05:13:53.482443+0000","last_clean":"2024-09-11T05:13:53.482443+0000","last_became_active":"2024-09-11T05:13:21.514919+0000","last_became_peered":"2024-09-11T05:13:21.514919+0000","last_unstale":"2024-09-11T05:13:53.482443+0000","last_undegraded":"2024-09-11T05:13:53.482443+0000","last_fullsized":"2024-09-11T05:13:53.482443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.714157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9715712,"num_objects":2372,"num_object_clones":0,"num_object_copies":4744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2257","reported_seq":2340,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.432277+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:13:52.432277+0000","last_peered":"2024-09-11T05:13:52.432277+0000","last_clean":"2024-09-11T05:13:52.432277+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:13:52.432277+0000","last_undegraded":"2024-09-11T05:13:52.432277+0000","last_fullsized":"2024-09-11T05:13:52.432277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":2257,"log_dups_size":0,"ondisk_log_size":2257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9220096,"num_objects":2251,"num_object_clones":0,"num_object_copies":4502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":655,"num_write_kb":2620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2170","reported_seq":2268,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.415742+0000","last_change":"2024-09-11T05:13:45.985545+0000","last_active":"2024-09-11T05:13:52.415742+0000","last_peered":"2024-09-11T05:13:52.415742+0000","last_clean":"2024-09-11T05:13:52.415742+0000","last_became_active":"2024-09-11T05:13:22.497399+0000","last_became_peered":"2024-09-11T05:13:22.497399+0000","last_unstale":"2024-09-11T05:13:52.415742+0000","last_undegraded":"2024-09-11T05:13:52.415742+0000","last_fullsized":"2024-09-11T05:13:52.415742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":2170,"log_dups_size":0,"ondisk_log_size":2170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:07.277685+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8871936,"num_objects":2166,"num_object_clones":0,"num_object_copies":4332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":573,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2279","reported_seq":2331,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.440542+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:13:52.440542+0000","last_peered":"2024-09-11T05:13:52.440542+0000","last_clean":"2024-09-11T05:13:52.440542+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:13:52.440542+0000","last_undegraded":"2024-09-11T05:13:52.440542+0000","last_fullsized":"2024-09-11T05:13:52.440542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2279,"log_dups_size":0,"ondisk_log_size":2279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9334784,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2279,"num_write_kb":9116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2355","reported_seq":2414,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480517+0000","last_change":"2024-09-11T05:13:21.517179+0000","last_active":"2024-09-11T05:13:53.480517+0000","last_peered":"2024-09-11T05:13:53.480517+0000","last_clean":"2024-09-11T05:12:42.440621+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:13:53.480517+0000","last_undegraded":"2024-09-11T05:13:21.496359+0000","last_fullsized":"2024-09-11T05:13:53.480517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2355,"log_dups_size":0,"ondisk_log_size":2355,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9618642,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2353,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2412","reported_seq":2470,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.456273+0000","last_change":"2024-09-11T05:13:21.514194+0000","last_active":"2024-09-11T05:13:53.456273+0000","last_peered":"2024-09-11T05:13:53.456273+0000","last_clean":"2024-09-11T05:12:43.818273+0000","last_became_active":"2024-09-11T05:13:21.512885+0000","last_became_peered":"2024-09-11T05:13:21.512885+0000","last_unstale":"2024-09-11T05:13:53.456273+0000","last_undegraded":"2024-09-11T05:13:21.492320+0000","last_fullsized":"2024-09-11T05:13:53.456273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2412,"log_dups_size":0,"ondisk_log_size":2412,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9860306,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2410,"num_write_kb":9631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2363","reported_seq":2421,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.456185+0000","last_change":"2024-09-11T05:13:21.511966+0000","last_active":"2024-09-11T05:13:53.456185+0000","last_peered":"2024-09-11T05:13:53.456185+0000","last_clean":"2024-09-11T05:12:42.812415+0000","last_became_active":"2024-09-11T05:13:21.511332+0000","last_became_peered":"2024-09-11T05:13:21.511332+0000","last_unstale":"2024-09-11T05:13:53.456185+0000","last_undegraded":"2024-09-11T05:13:21.492627+0000","last_fullsized":"2024-09-11T05:13:53.456185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9655506,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":9432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2312","reported_seq":2375,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-11T05:13:53.478260+0000","last_change":"2024-09-11T05:13:22.493892+0000","last_active":"2024-09-11T05:13:53.478260+0000","last_peered":"2024-09-11T05:13:53.478260+0000","last_clean":"2024-09-11T05:12:33.173961+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:13:53.478260+0000","last_undegraded":"2024-09-11T05:13:53.478260+0000","last_fullsized":"2024-09-11T05:13:53.478260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462377,"num_objects":2311,"num_object_clones":0,"num_object_copies":4622,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2311,"num_write_kb":9241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.493688+0000","last_change":"2024-09-11T05:13:21.508556+0000","last_active":"2024-09-11T05:13:22.493688+0000","last_peered":"2024-09-11T05:13:22.493688+0000","last_clean":"2024-09-11T05:13:22.493688+0000","last_became_active":"2024-09-11T05:13:21.507784+0000","last_became_peered":"2024-09-11T05:13:21.507784+0000","last_unstale":"2024-09-11T05:13:22.493688+0000","last_undegraded":"2024-09-11T05:13:22.493688+0000","last_fullsized":"2024-09-11T05:13:22.493688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:10.209768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2256","reported_seq":2362,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.456494+0000","last_change":"2024-09-11T05:13:49.372903+0000","last_active":"2024-09-11T05:13:52.456494+0000","last_peered":"2024-09-11T05:13:52.456494+0000","last_clean":"2024-09-11T05:13:52.456494+0000","last_became_active":"2024-09-11T05:13:22.496902+0000","last_became_peered":"2024-09-11T05:13:22.496902+0000","last_unstale":"2024-09-11T05:13:52.456494+0000","last_undegraded":"2024-09-11T05:13:52.456494+0000","last_fullsized":"2024-09-11T05:13:52.456494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":2256,"log_dups_size":0,"ondisk_log_size":2256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:47.496372+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9211904,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":1296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497168+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:13:22.497168+0000","last_peered":"2024-09-11T05:13:22.497168+0000","last_clean":"2024-09-11T05:13:22.497168+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:13:22.497168+0000","last_undegraded":"2024-09-11T05:13:22.497168+0000","last_fullsized":"2024-09-11T05:13:22.497168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2240","reported_seq":2291,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.472695+0000","last_change":"2024-09-11T05:13:21.514035+0000","last_active":"2024-09-11T05:13:53.472695+0000","last_peered":"2024-09-11T05:13:53.472695+0000","last_clean":"2024-09-11T05:12:43.902687+0000","last_became_active":"2024-09-11T05:13:21.511727+0000","last_became_peered":"2024-09-11T05:13:21.511727+0000","last_unstale":"2024-09-11T05:13:53.472695+0000","last_undegraded":"2024-09-11T05:13:21.493049+0000","last_fullsized":"2024-09-11T05:13:53.472695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9143506,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2238,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":69,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:22.493699+0000","last_change":"2024-09-11T05:13:21.513836+0000","last_active":"2024-09-11T05:13:22.493699+0000","last_peered":"2024-09-11T05:13:22.493699+0000","last_clean":"2024-09-11T05:12:55.450586+0000","last_became_active":"2024-09-11T05:13:21.506029+0000","last_became_peered":"2024-09-11T05:13:21.506029+0000","last_unstale":"2024-09-11T05:13:22.493699+0000","last_undegraded":"2024-09-11T05:13:21.491804+0000","last_fullsized":"2024-09-11T05:13:22.493699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2193","reported_seq":2285,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.432064+0000","last_change":"2024-09-11T05:13:49.998432+0000","last_active":"2024-09-11T05:13:52.432064+0000","last_peered":"2024-09-11T05:13:52.432064+0000","last_clean":"2024-09-11T05:13:52.432064+0000","last_became_active":"2024-09-11T05:13:22.495923+0000","last_became_peered":"2024-09-11T05:13:22.495923+0000","last_unstale":"2024-09-11T05:13:52.432064+0000","last_undegraded":"2024-09-11T05:13:52.432064+0000","last_fullsized":"2024-09-11T05:13:52.432064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":2193,"log_dups_size":0,"ondisk_log_size":2193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:19.484483+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8962048,"num_objects":2188,"num_object_clones":0,"num_object_copies":4376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505097+0000","last_change":"2024-09-11T05:13:22.497695+0000","last_active":"2024-09-11T05:13:22.505097+0000","last_peered":"2024-09-11T05:13:22.505097+0000","last_clean":"2024-09-11T05:13:22.505097+0000","last_became_active":"2024-09-11T05:13:22.497288+0000","last_became_peered":"2024-09-11T05:13:22.497288+0000","last_unstale":"2024-09-11T05:13:22.505097+0000","last_undegraded":"2024-09-11T05:13:22.505097+0000","last_fullsized":"2024-09-11T05:13:22.505097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:56.694811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2234","reported_seq":2327,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:52.443057+0000","last_change":"2024-09-11T05:13:51.225829+0000","last_active":"2024-09-11T05:13:52.443057+0000","last_peered":"2024-09-11T05:13:52.443057+0000","last_clean":"2024-09-11T05:13:52.443057+0000","last_became_active":"2024-09-11T05:13:22.496829+0000","last_became_peered":"2024-09-11T05:13:22.496829+0000","last_unstale":"2024-09-11T05:13:52.443057+0000","last_undegraded":"2024-09-11T05:13:52.443057+0000","last_fullsized":"2024-09-11T05:13:52.443057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:27.126916+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9125888,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":137,"num_write_kb":548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2348","reported_seq":2406,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.455822+0000","last_change":"2024-09-11T05:13:21.512469+0000","last_active":"2024-09-11T05:13:53.455822+0000","last_peered":"2024-09-11T05:13:53.455822+0000","last_clean":"2024-09-11T05:12:35.198001+0000","last_became_active":"2024-09-11T05:13:21.511812+0000","last_became_peered":"2024-09-11T05:13:21.511812+0000","last_unstale":"2024-09-11T05:13:53.455822+0000","last_undegraded":"2024-09-11T05:13:21.496652+0000","last_fullsized":"2024-09-11T05:13:53.455822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2348,"log_dups_size":0,"ondisk_log_size":2348,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9609833,"num_objects":2347,"num_object_clones":0,"num_object_copies":4694,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2347,"num_write_kb":9385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2347}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.494954+0000","last_change":"2024-09-11T05:13:21.512849+0000","last_active":"2024-09-11T05:13:22.494954+0000","last_peered":"2024-09-11T05:13:22.494954+0000","last_clean":"2024-09-11T05:13:22.494954+0000","last_became_active":"2024-09-11T05:13:21.512089+0000","last_became_peered":"2024-09-11T05:13:21.512089+0000","last_unstale":"2024-09-11T05:13:22.494954+0000","last_undegraded":"2024-09-11T05:13:22.494954+0000","last_fullsized":"2024-09-11T05:13:22.494954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:09:25.087655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2261","reported_seq":2318,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480418+0000","last_change":"2024-09-11T05:13:21.516438+0000","last_active":"2024-09-11T05:13:53.480418+0000","last_peered":"2024-09-11T05:13:53.480418+0000","last_clean":"2024-09-11T05:12:43.732555+0000","last_became_active":"2024-09-11T05:13:21.515407+0000","last_became_peered":"2024-09-11T05:13:21.515407+0000","last_unstale":"2024-09-11T05:13:53.480418+0000","last_undegraded":"2024-09-11T05:13:21.496548+0000","last_fullsized":"2024-09-11T05:13:53.480418+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9230139,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2258,"num_write_kb":9016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2256}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497841+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:13:22.497841+0000","last_peered":"2024-09-11T05:13:22.497841+0000","last_clean":"2024-09-11T05:13:22.497841+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:13:22.497841+0000","last_undegraded":"2024-09-11T05:13:22.497841+0000","last_fullsized":"2024-09-11T05:13:22.497841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2310","reported_seq":2367,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.472969+0000","last_change":"2024-09-11T05:13:21.516118+0000","last_active":"2024-09-11T05:13:53.472969+0000","last_peered":"2024-09-11T05:13:53.472969+0000","last_clean":"2024-09-11T05:12:35.198039+0000","last_became_active":"2024-09-11T05:13:21.515226+0000","last_became_peered":"2024-09-11T05:13:21.515226+0000","last_unstale":"2024-09-11T05:13:53.472969+0000","last_undegraded":"2024-09-11T05:13:21.494389+0000","last_fullsized":"2024-09-11T05:13:53.472969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9446610,"num_objects":2308,"num_object_clones":0,"num_object_copies":4616,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2308,"num_write_kb":9226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2308}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495153+0000","last_change":"2024-09-11T05:13:21.511859+0000","last_active":"2024-09-11T05:13:22.495153+0000","last_peered":"2024-09-11T05:13:22.495153+0000","last_clean":"2024-09-11T05:13:22.495153+0000","last_became_active":"2024-09-11T05:13:21.511091+0000","last_became_peered":"2024-09-11T05:13:21.511091+0000","last_unstale":"2024-09-11T05:13:22.495153+0000","last_undegraded":"2024-09-11T05:13:22.495153+0000","last_fullsized":"2024-09-11T05:13:22.495153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:05.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505152+0000","last_change":"2024-09-11T05:13:22.498193+0000","last_active":"2024-09-11T05:13:22.505152+0000","last_peered":"2024-09-11T05:13:22.505152+0000","last_clean":"2024-09-11T05:13:22.505152+0000","last_became_active":"2024-09-11T05:13:22.498019+0000","last_became_peered":"2024-09-11T05:13:22.498019+0000","last_unstale":"2024-09-11T05:13:22.505152+0000","last_undegraded":"2024-09-11T05:13:22.505152+0000","last_fullsized":"2024-09-11T05:13:22.505152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:24.352250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2307","reported_seq":2364,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T05:13:53.480656+0000","last_change":"2024-09-11T05:13:21.513883+0000","last_active":"2024-09-11T05:13:53.480656+0000","last_peered":"2024-09-11T05:13:53.480656+0000","last_clean":"2024-09-11T05:12:43.318518+0000","last_became_active":"2024-09-11T05:13:21.512960+0000","last_became_peered":"2024-09-11T05:13:21.512960+0000","last_unstale":"2024-09-11T05:13:53.480656+0000","last_undegraded":"2024-09-11T05:13:21.493149+0000","last_fullsized":"2024-09-11T05:13:53.480656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2307,"log_dups_size":0,"ondisk_log_size":2307,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9402788,"num_objects":2299,"num_object_clones":0,"num_object_copies":4598,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2304,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149872075,"num_objects":36606,"num_object_clones":0,"num_object_copies":73212,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27452,"num_write_kb":109694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298520576,"data_stored":298402290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36669,"ondisk_log_size":36669,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232376,"kb_used_data":142376,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94139464,"statfs":{"total":96636764160,"available":96398811136,"internally_reserved":0,"allocated":145793024,"data_stored":145585448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":92139179},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215121,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254996,"kb_used_data":147008,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94116844,"statfs":{"total":96636764160,"available":96375648256,"internally_reserved":0,"allocated":150536192,"data_stored":150303665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110554795},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,18,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151957504,"data_stored":151936630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146448384,"data_stored":146448384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:13:57.074 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 24 20 41762 41742 6.79088 5.14453 0.0155483 0.0114981 2024-09-11T05:13:58.074 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 25 20 43130 43110 6.733 5.34375 0.00883942 0.0115971 2024-09-11T05:13:59.074 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 26 20 44408 44388 6.66606 4.99219 0.0256124 0.011708 2024-09-11T05:13:59.728 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:00.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 27 20 45628 45608 6.59568 4.76562 0.0199338 0.0118356 2024-09-11T05:14:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:00.305 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-09-11T05:13:59.368603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173030878,"num_objects":42262,"num_object_clones":0,"num_object_copies":84524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33108,"num_write_kb":132312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":636,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42326,"ondisk_log_size":42326,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":543152,"kb_used_data":331280,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188200528,"statfs":{"total":193273528320,"available":192717340672,"internally_reserved":0,"allocated":339230720,"data_stored":338808012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8874,"internal_metadata":216915286},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,6,4,10,9,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":73482017,"num_objects":17934,"num_object_clones":0,"num_object_copies":35868,"num_objects_missing_on_primary":-24,"num_objects_missing":-24,"num_objects_degraded":-48,"num_objects_misplaced":0,"num_objects_unfound":-24,"num_objects_dirty":17934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":41452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":636,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002617"},"pg_stats":[{"pgid":"2.f","version":"36'2821","reported_seq":2872,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.506704+0000","last_change":"2024-09-11T05:13:21.515668+0000","last_active":"2024-09-11T05:13:58.506704+0000","last_peered":"2024-09-11T05:13:58.506704+0000","last_clean":"2024-09-11T05:13:58.506704+0000","last_became_active":"2024-09-11T05:13:21.514919+0000","last_became_peered":"2024-09-11T05:13:21.514919+0000","last_unstale":"2024-09-11T05:13:58.506704+0000","last_undegraded":"2024-09-11T05:13:58.506704+0000","last_fullsized":"2024-09-11T05:13:58.506704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2821,"log_dups_size":0,"ondisk_log_size":2821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.714157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11554816,"num_objects":2821,"num_object_clones":0,"num_object_copies":5642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2821,"num_write_kb":11284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2490","reported_seq":2573,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:55.441334+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:13:55.441334+0000","last_peered":"2024-09-11T05:13:55.441334+0000","last_clean":"2024-09-11T05:13:55.441334+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:13:55.441334+0000","last_undegraded":"2024-09-11T05:13:55.441334+0000","last_fullsized":"2024-09-11T05:13:55.441334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":2490,"log_dups_size":0,"ondisk_log_size":2490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10174464,"num_objects":2484,"num_object_clones":0,"num_object_copies":4968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2426","reported_seq":2524,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:55.434011+0000","last_change":"2024-09-11T05:13:45.985545+0000","last_active":"2024-09-11T05:13:55.434011+0000","last_peered":"2024-09-11T05:13:55.434011+0000","last_clean":"2024-09-11T05:13:55.434011+0000","last_became_active":"2024-09-11T05:13:22.497399+0000","last_became_peered":"2024-09-11T05:13:22.497399+0000","last_unstale":"2024-09-11T05:13:55.434011+0000","last_undegraded":"2024-09-11T05:13:55.434011+0000","last_fullsized":"2024-09-11T05:13:55.434011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":2426,"log_dups_size":0,"ondisk_log_size":2426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:07.277685+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9920512,"num_objects":2422,"num_object_clones":0,"num_object_copies":4844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":3316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2546","reported_seq":2598,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:55.447280+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:13:55.447280+0000","last_peered":"2024-09-11T05:13:55.447280+0000","last_clean":"2024-09-11T05:13:55.447280+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:13:55.447280+0000","last_undegraded":"2024-09-11T05:13:55.447280+0000","last_fullsized":"2024-09-11T05:13:55.447280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2546,"log_dups_size":0,"ondisk_log_size":2546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10428416,"num_objects":2546,"num_object_clones":0,"num_object_copies":5092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2793","reported_seq":2864,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.501546+0000","last_change":"2024-09-11T05:13:57.829585+0000","last_active":"2024-09-11T05:13:58.501546+0000","last_peered":"2024-09-11T05:13:58.501546+0000","last_clean":"2024-09-11T05:13:58.501546+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:13:58.501546+0000","last_undegraded":"2024-09-11T05:13:58.501546+0000","last_fullsized":"2024-09-11T05:13:58.501546+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2793,"log_dups_size":0,"ondisk_log_size":2793,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11412690,"num_objects":2788,"num_object_clones":0,"num_object_copies":5576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2791,"num_write_kb":11148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2834","reported_seq":2903,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.497884+0000","last_change":"2024-09-11T05:13:57.619810+0000","last_active":"2024-09-11T05:13:58.497884+0000","last_peered":"2024-09-11T05:13:58.497884+0000","last_clean":"2024-09-11T05:13:58.497884+0000","last_became_active":"2024-09-11T05:13:21.512885+0000","last_became_peered":"2024-09-11T05:13:21.512885+0000","last_unstale":"2024-09-11T05:13:58.497884+0000","last_undegraded":"2024-09-11T05:13:58.497884+0000","last_fullsized":"2024-09-11T05:13:58.497884+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2834,"log_dups_size":0,"ondisk_log_size":2834,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11588818,"num_objects":2831,"num_object_clones":0,"num_object_copies":5662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2832,"num_write_kb":11319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2744","reported_seq":2813,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.461043+0000","last_change":"2024-09-11T05:13:57.300235+0000","last_active":"2024-09-11T05:13:58.461043+0000","last_peered":"2024-09-11T05:13:58.461043+0000","last_clean":"2024-09-11T05:13:58.461043+0000","last_became_active":"2024-09-11T05:13:21.511332+0000","last_became_peered":"2024-09-11T05:13:21.511332+0000","last_unstale":"2024-09-11T05:13:58.461043+0000","last_undegraded":"2024-09-11T05:13:58.461043+0000","last_fullsized":"2024-09-11T05:13:58.461043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2744,"log_dups_size":0,"ondisk_log_size":2744,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11216082,"num_objects":2740,"num_object_clones":0,"num_object_copies":5480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2742,"num_write_kb":10956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2689","reported_seq":2766,"reported_epoch":36,"state":"active+clean+scrubbing","last_fresh":"2024-09-11T05:13:58.489533+0000","last_change":"2024-09-11T05:13:58.194530+0000","last_active":"2024-09-11T05:13:58.489533+0000","last_peered":"2024-09-11T05:13:58.489533+0000","last_clean":"2024-09-11T05:13:58.489533+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:13:58.489533+0000","last_undegraded":"2024-09-11T05:13:58.489533+0000","last_fullsized":"2024-09-11T05:13:58.489533+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":699,"log_size":2689,"log_dups_size":0,"ondisk_log_size":2689,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11006569,"num_objects":2688,"num_object_clones":0,"num_object_copies":5376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2688,"num_write_kb":10749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.493688+0000","last_change":"2024-09-11T05:13:21.508556+0000","last_active":"2024-09-11T05:13:22.493688+0000","last_peered":"2024-09-11T05:13:22.493688+0000","last_clean":"2024-09-11T05:13:22.493688+0000","last_became_active":"2024-09-11T05:13:21.507784+0000","last_became_peered":"2024-09-11T05:13:21.507784+0000","last_unstale":"2024-09-11T05:13:22.493688+0000","last_undegraded":"2024-09-11T05:13:22.493688+0000","last_fullsized":"2024-09-11T05:13:22.493688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:10.209768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2484","reported_seq":2590,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:55.434207+0000","last_change":"2024-09-11T05:13:49.372903+0000","last_active":"2024-09-11T05:13:55.434207+0000","last_peered":"2024-09-11T05:13:55.434207+0000","last_clean":"2024-09-11T05:13:55.434207+0000","last_became_active":"2024-09-11T05:13:22.496902+0000","last_became_peered":"2024-09-11T05:13:22.496902+0000","last_unstale":"2024-09-11T05:13:55.434207+0000","last_undegraded":"2024-09-11T05:13:55.434207+0000","last_fullsized":"2024-09-11T05:13:55.434207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":2484,"log_dups_size":0,"ondisk_log_size":2484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:47.496372+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10145792,"num_objects":2477,"num_object_clones":0,"num_object_copies":4954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":552,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497168+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:13:22.497168+0000","last_peered":"2024-09-11T05:13:22.497168+0000","last_clean":"2024-09-11T05:13:22.497168+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:13:22.497168+0000","last_undegraded":"2024-09-11T05:13:22.497168+0000","last_fullsized":"2024-09-11T05:13:22.497168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2654","reported_seq":2714,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.510986+0000","last_change":"2024-09-11T05:13:57.546543+0000","last_active":"2024-09-11T05:13:58.510986+0000","last_peered":"2024-09-11T05:13:58.510986+0000","last_clean":"2024-09-11T05:13:58.510986+0000","last_became_active":"2024-09-11T05:13:21.511727+0000","last_became_peered":"2024-09-11T05:13:21.511727+0000","last_unstale":"2024-09-11T05:13:58.510986+0000","last_undegraded":"2024-09-11T05:13:58.510986+0000","last_fullsized":"2024-09-11T05:13:58.510986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2654,"log_dups_size":0,"ondisk_log_size":2654,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10839250,"num_objects":2648,"num_object_clones":0,"num_object_copies":5296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2652,"num_write_kb":10588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":76,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:57.852414+0000","last_change":"2024-09-11T05:13:57.850191+0000","last_active":"2024-09-11T05:13:57.852414+0000","last_peered":"2024-09-11T05:13:57.852414+0000","last_clean":"2024-09-11T05:13:57.852414+0000","last_became_active":"2024-09-11T05:13:21.506029+0000","last_became_peered":"2024-09-11T05:13:21.506029+0000","last_unstale":"2024-09-11T05:13:57.852414+0000","last_undegraded":"2024-09-11T05:13:57.852414+0000","last_fullsized":"2024-09-11T05:13:57.852414+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:07.141905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2428","reported_seq":2518,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:55.448440+0000","last_change":"2024-09-11T05:13:49.998432+0000","last_active":"2024-09-11T05:13:55.448440+0000","last_peered":"2024-09-11T05:13:55.448440+0000","last_clean":"2024-09-11T05:13:55.448440+0000","last_became_active":"2024-09-11T05:13:22.495923+0000","last_became_peered":"2024-09-11T05:13:22.495923+0000","last_unstale":"2024-09-11T05:13:55.448440+0000","last_undegraded":"2024-09-11T05:13:55.448440+0000","last_fullsized":"2024-09-11T05:13:55.448440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":2428,"log_dups_size":0,"ondisk_log_size":2428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:19.484483+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9924608,"num_objects":2423,"num_object_clones":0,"num_object_copies":4846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":1828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505097+0000","last_change":"2024-09-11T05:13:22.497695+0000","last_active":"2024-09-11T05:13:22.505097+0000","last_peered":"2024-09-11T05:13:22.505097+0000","last_clean":"2024-09-11T05:13:22.505097+0000","last_became_active":"2024-09-11T05:13:22.497288+0000","last_became_peered":"2024-09-11T05:13:22.497288+0000","last_unstale":"2024-09-11T05:13:22.505097+0000","last_undegraded":"2024-09-11T05:13:22.505097+0000","last_fullsized":"2024-09-11T05:13:22.505097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:56.694811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2481","reported_seq":2574,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:55.423862+0000","last_change":"2024-09-11T05:13:51.225829+0000","last_active":"2024-09-11T05:13:55.423862+0000","last_peered":"2024-09-11T05:13:55.423862+0000","last_clean":"2024-09-11T05:13:55.423862+0000","last_became_active":"2024-09-11T05:13:22.496829+0000","last_became_peered":"2024-09-11T05:13:22.496829+0000","last_unstale":"2024-09-11T05:13:55.423862+0000","last_undegraded":"2024-09-11T05:13:55.423862+0000","last_fullsized":"2024-09-11T05:13:55.423862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":2481,"log_dups_size":0,"ondisk_log_size":2481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:27.126916+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10137600,"num_objects":2475,"num_object_clones":0,"num_object_copies":4950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2778","reported_seq":2844,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.498118+0000","last_change":"2024-09-11T05:13:57.325488+0000","last_active":"2024-09-11T05:13:58.498118+0000","last_peered":"2024-09-11T05:13:58.498118+0000","last_clean":"2024-09-11T05:13:58.498118+0000","last_became_active":"2024-09-11T05:13:21.511812+0000","last_became_peered":"2024-09-11T05:13:21.511812+0000","last_unstale":"2024-09-11T05:13:58.498118+0000","last_undegraded":"2024-09-11T05:13:58.498118+0000","last_fullsized":"2024-09-11T05:13:58.498118+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11371113,"num_objects":2777,"num_object_clones":0,"num_object_copies":5554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2777,"num_write_kb":11105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.494954+0000","last_change":"2024-09-11T05:13:21.512849+0000","last_active":"2024-09-11T05:13:22.494954+0000","last_peered":"2024-09-11T05:13:22.494954+0000","last_clean":"2024-09-11T05:13:22.494954+0000","last_became_active":"2024-09-11T05:13:21.512089+0000","last_became_peered":"2024-09-11T05:13:21.512089+0000","last_unstale":"2024-09-11T05:13:22.494954+0000","last_undegraded":"2024-09-11T05:13:22.494954+0000","last_fullsized":"2024-09-11T05:13:22.494954+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:09:25.087655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2686","reported_seq":2757,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.507387+0000","last_change":"2024-09-11T05:13:57.764568+0000","last_active":"2024-09-11T05:13:58.507387+0000","last_peered":"2024-09-11T05:13:58.507387+0000","last_clean":"2024-09-11T05:13:58.507387+0000","last_became_active":"2024-09-11T05:13:21.515407+0000","last_became_peered":"2024-09-11T05:13:21.515407+0000","last_unstale":"2024-09-11T05:13:58.507387+0000","last_undegraded":"2024-09-11T05:13:58.507387+0000","last_fullsized":"2024-09-11T05:13:58.507387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2686,"log_dups_size":0,"ondisk_log_size":2686,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10970939,"num_objects":2681,"num_object_clones":0,"num_object_copies":5362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2683,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.497841+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:13:22.497841+0000","last_peered":"2024-09-11T05:13:22.497841+0000","last_clean":"2024-09-11T05:13:22.497841+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:13:22.497841+0000","last_undegraded":"2024-09-11T05:13:22.497841+0000","last_fullsized":"2024-09-11T05:13:22.497841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2722","reported_seq":2790,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.481358+0000","last_change":"2024-09-11T05:13:57.691884+0000","last_active":"2024-09-11T05:13:58.481358+0000","last_peered":"2024-09-11T05:13:58.481358+0000","last_clean":"2024-09-11T05:13:58.481358+0000","last_became_active":"2024-09-11T05:13:21.515226+0000","last_became_peered":"2024-09-11T05:13:21.515226+0000","last_unstale":"2024-09-11T05:13:58.481358+0000","last_undegraded":"2024-09-11T05:13:58.481358+0000","last_fullsized":"2024-09-11T05:13:58.481358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2722,"log_dups_size":0,"ondisk_log_size":2722,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11134162,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2720,"num_write_kb":10874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.495153+0000","last_change":"2024-09-11T05:13:21.511859+0000","last_active":"2024-09-11T05:13:22.495153+0000","last_peered":"2024-09-11T05:13:22.495153+0000","last_clean":"2024-09-11T05:13:22.495153+0000","last_became_active":"2024-09-11T05:13:21.511091+0000","last_became_peered":"2024-09-11T05:13:21.511091+0000","last_unstale":"2024-09-11T05:13:22.495153+0000","last_undegraded":"2024-09-11T05:13:22.495153+0000","last_fullsized":"2024-09-11T05:13:22.495153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:59:05.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:22.505152+0000","last_change":"2024-09-11T05:13:22.498193+0000","last_active":"2024-09-11T05:13:22.505152+0000","last_peered":"2024-09-11T05:13:22.505152+0000","last_clean":"2024-09-11T05:13:22.505152+0000","last_became_active":"2024-09-11T05:13:22.498019+0000","last_became_peered":"2024-09-11T05:13:22.498019+0000","last_unstale":"2024-09-11T05:13:22.505152+0000","last_undegraded":"2024-09-11T05:13:22.505152+0000","last_fullsized":"2024-09-11T05:13:22.505152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:37:24.352250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2747","reported_seq":2821,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T05:13:58.476959+0000","last_change":"2024-09-11T05:13:57.444640+0000","last_active":"2024-09-11T05:13:58.476959+0000","last_peered":"2024-09-11T05:13:58.476959+0000","last_clean":"2024-09-11T05:13:58.476959+0000","last_became_active":"2024-09-11T05:13:21.512960+0000","last_became_peered":"2024-09-11T05:13:21.512960+0000","last_unstale":"2024-09-11T05:13:58.476959+0000","last_undegraded":"2024-09-11T05:13:58.476959+0000","last_fullsized":"2024-09-11T05:13:58.476959+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":2747,"log_dups_size":0,"ondisk_log_size":2747,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11205028,"num_objects":2739,"num_object_clones":0,"num_object_copies":5478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":617,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173030859,"num_objects":42260,"num_object_clones":0,"num_object_copies":84520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33106,"num_write_kb":132310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":617,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":341733376,"data_stored":341632485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42323,"ondisk_log_size":42323,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":270024,"kb_used_data":157816,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94101816,"statfs":{"total":96636764160,"available":96360259584,"internally_reserved":0,"allocated":161603584,"data_stored":161396008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":114880171},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215122,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":273128,"kb_used_data":173464,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94098712,"statfs":{"total":96636764160,"available":96357081088,"internally_reserved":0,"allocated":177627136,"data_stored":177412004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":102035115},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,6,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179138560,"data_stored":179135081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162480128,"data_stored":162480128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:00.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-11T05:14:00.368 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-09-11T05:14:00.368 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-09-11T05:14:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-09-11T05:14:00.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:00.465 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-09-11T05:14:00.465 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-09-11T05:14:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-09-11T05:14:00.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:00.564 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-09-11T05:14:00.564 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-09-11T05:14:00.655 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-09-11T05:14:00.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:00.659 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-09-11T05:14:00.659 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-09-11T05:14:00.759 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-09-11T05:14:00.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:00.765 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-09-11T05:14:00.766 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-09-11T05:14:00.849 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-09-11T05:14:00.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:00.854 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-09-11T05:14:00.855 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-09-11T05:14:00.946 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-09-11T05:14:00.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:00.953 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-09-11T05:14:00.954 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-09-11T05:14:01.028 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-09-11T05:14:01.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.039 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-09-11T05:14:01.039 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-09-11T05:14:01.074 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 28 20 46766 46746 6.51889 4.44531 0.0107154 0.0119752 2024-09-11T05:14:01.112 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-09-11T05:14:01.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.118 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-09-11T05:14:01.118 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-09-11T05:14:01.247 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-09-11T05:14:01.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.250 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-09-11T05:14:01.250 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-09-11T05:14:01.328 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-09-11T05:14:01.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.339 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-09-11T05:14:01.339 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-09-11T05:14:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-09-11T05:14:01.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.419 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-09-11T05:14:01.419 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-09-11T05:14:01.515 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-09-11T05:14:01.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.519 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-09-11T05:14:01.519 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-09-11T05:14:01.598 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-09-11T05:14:01.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.602 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-09-11T05:14:01.602 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-09-11T05:14:01.693 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-09-11T05:14:01.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.700 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-09-11T05:14:01.700 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-09-11T05:14:01.793 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-09-11T05:14:01.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.798 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-09-11T05:14:01.798 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-09-11T05:14:01.897 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-09-11T05:14:01.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.900 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-09-11T05:14:01.900 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-09-11T05:14:01.963 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-09-11T05:14:01.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:01.966 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-09-11T05:14:01.966 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-09-11T05:14:02.056 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-09-11T05:14:02.075 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 29 20 47825 47805 6.43676 4.13672 0.0283525 0.0121312 2024-09-11T05:14:02.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.081 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-09-11T05:14:02.081 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-09-11T05:14:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-09-11T05:14:02.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.183 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-09-11T05:14:02.184 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-09-11T05:14:02.268 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-09-11T05:14:02.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.279 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-09-11T05:14:02.280 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-09-11T05:14:02.357 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-09-11T05:14:02.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.372 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-09-11T05:14:02.372 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-09-11T05:14:02.453 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-09-11T05:14:02.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-11T05:14:02.452+0000 7f8e6aed1640 -1 monclient: get_auth_request but no auth handler is set up 2024-09-11T05:14:02.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.460 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-09-11T05:14:02.461 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-09-11T05:14:02.536 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-09-11T05:14:02.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.545 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-09-11T05:14:02.545 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-09-11T05:14:02.629 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-09-11T05:14:02.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.636 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-09-11T05:14:02.636 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-09-11T05:14:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-09-11T05:14:02.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.718 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-09-11T05:14:02.718 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-09-11T05:14:02.780 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-09-11T05:14:02.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.786 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-09-11T05:14:02.787 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-09-11T05:14:02.848 INFO:teuthology.orchestra.run.smithi133.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-09-11T05:14:02.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:14:02.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-11T05:14:03.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 30 20 48923 48903 6.36496 4.28906 0.0180161 0.012267 2024-09-11T05:14:03.458 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-09-11T05:14:03.456+0000 7f9b75702640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-11T05:14:03.682 INFO:teuthology.orchestra.run.smithi133.stderr:marked in osd.1. 2024-09-11T05:14:03.707 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T05:14:03.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T05:14:03.709 INFO:tasks.ceph.osd.1:Started 2024-09-11T05:14:03.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T05:14:03.806 INFO:teuthology.orchestra.run.smithi133.stderr:no valid command found; 10 closest matches: 2024-09-11T05:14:03.806 INFO:teuthology.orchestra.run.smithi133.stderr:0 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:1 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:2 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:abort 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:assert 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:config diff 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:config diff get 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:config get 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:config help [] 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:config set ... 2024-09-11T05:14:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:admin_socket: invalid command 2024-09-11T05:14:03.808 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T05:14:03.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-11T05:14:04.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 31 20 49620 49600 6.24748 2.72266 0.0170028 0.0123415 2024-09-11T05:14:05.036 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-09-11T05:14:05.032+0000 7efee99e77c0 -1 Falling back to public interface 2024-09-11T05:14:05.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 32 20 49620 49600 6.0523 0 - 0.0123415 2024-09-11T05:14:06.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 33 20 50042 50022 5.91888 0.824219 0.00650256 0.0131923 2024-09-11T05:14:07.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 34 20 51210 51190 5.87899 4.5625 0.0120289 0.0132829 2024-09-11T05:14:08.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 35 20 52315 52295 5.83435 4.31641 0.0246698 0.013384 2024-09-11T05:14:08.810 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T05:14:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:no valid command found; 10 closest matches: 2024-09-11T05:14:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:0 2024-09-11T05:14:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:1 2024-09-11T05:14:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:2 2024-09-11T05:14:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:abort 2024-09-11T05:14:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:assert 2024-09-11T05:14:09.011 INFO:teuthology.orchestra.run.smithi133.stderr:config diff 2024-09-11T05:14:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:config diff get 2024-09-11T05:14:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:config get 2024-09-11T05:14:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:config help [] 2024-09-11T05:14:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:config set ... 2024-09-11T05:14:09.012 INFO:teuthology.orchestra.run.smithi133.stderr:admin_socket: invalid command 2024-09-11T05:14:09.013 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T05:14:09.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-11T05:14:09.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 36 20 53350 53330 5.78459 4.04297 0.00918007 0.0134969 2024-09-11T05:14:09.802 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-09-11T05:14:09.796+0000 7efee99e77c0 -1 osd.1 25 log_to_monitors true 2024-09-11T05:14:10.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 37 20 54434 54414 5.74269 4.23438 0.0133399 0.0135989 2024-09-11T05:14:10.723 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-09-11T05:14:10.720+0000 7efee17ca640 -1 osd.1 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T05:14:11.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 38 20 55560 55540 5.70732 4.39844 0.0148407 0.0136828 2024-09-11T05:14:12.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 39 20 56709 56689 5.67606 4.48828 0.00897651 0.0137552 2024-09-11T05:14:13.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:14:13.078781+0000 min lat: 0.00201616 max lat: 2.10366 avg lat: 0.0137621 2024-09-11T05:14:13.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:14:13.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 40 20 56787 56767 5.5418 0.304688 0.0157841 0.0137621 2024-09-11T05:14:14.015 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T05:14:14.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 41 20 56787 56767 5.40666 0 - 0.0137621 2024-09-11T05:14:14.215 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:14:14.215 INFO:teuthology.orchestra.run.smithi133.stdout: "ops": [ 2024-09-11T05:14:14.215 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.215 INFO:teuthology.orchestra.run.smithi133.stdout: "description": "osd_repop(client.5204.0:56792 2.0 e43/42 2:0232bdc1:::benchmark_data_smithi133_33840_object56791:head v 43'3486, mlcod=22'4)", 2024-09-11T05:14:14.215 INFO:teuthology.orchestra.run.smithi133.stdout: "initiated_at": "2024-09-11T05:14:14.207802+0000", 2024-09-11T05:14:14.215 INFO:teuthology.orchestra.run.smithi133.stdout: "age": 0.0083832160000000006, 2024-09-11T05:14:14.215 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 0.008408357, 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "continuous": false, 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "type_data": { 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "flag_point": "started", 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "events": [ 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "initiated", 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.207802+0000", 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 0 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "header_read", 2024-09-11T05:14:14.216 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.207802+0000", 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 0 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "throttled", 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.207805+0000", 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 2.3180000000000001e-06 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "all_read", 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.207819+0000", 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 1.4117999999999999e-05 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:14:14.217 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "dispatched", 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.209985+0000", 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 0.002166074 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "queued_for_pg", 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.210000+0000", 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 1.5647000000000001e-05 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "reached_pg", 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.210060+0000", 2024-09-11T05:14:14.218 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 5.9726999999999999e-05 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: "event": "started", 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: "time": "2024-09-11T05:14:14.210087+0000", 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: "duration": 2.7311e-05 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: "num_ops": 1 2024-09-11T05:14:14.219 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:14:14.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-09-11T05:14:14.242 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:14:14.934 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:14.934 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":43,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:14:13.727748+0000","last_up_change":"2024-09-11T05:14:11.716852+0000","last_in_change":"2024-09-11T05:14:03.662755+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T05:12:17.695239+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T05:12:29.395526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":24,"up_thru":42,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":3512043189}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":3512043189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":3512043189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":3512043189}]},"public_addr":"172.21.15.133:6804/3512043189","cluster_addr":"172.21.15.133:6805/3512043189","heartbeat_back_addr":"172.21.15.133:6807/3512043189","heartbeat_front_addr":"172.21.15.133:6806/3512043189","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":25,"up_from":41,"up_thru":42,"down_at":26,"lost_at":26,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":3859356444}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":3859356444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":3859356444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":3859356444}]},"public_addr":"172.21.15.133:6808/3859356444","cluster_addr":"172.21.15.133:6809/3859356444","heartbeat_back_addr":"172.21.15.133:6811/3859356444","heartbeat_front_addr":"172.21.15.133:6810/3859356444","state":["exists","up"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":42,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":401696059}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":401696059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":401696059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":401696059}]},"public_addr":"172.21.15.133:6800/401696059","cluster_addr":"172.21.15.133:6801/401696059","heartbeat_back_addr":"172.21.15.133:6803/401696059","heartbeat_front_addr":"172.21.15.133:6802/401696059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":26},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":18}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:14:15.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 42 20 57292 57272 5.32492 0.986328 0.0360438 0.0146633 2024-09-11T05:14:15.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-09-11T05:14:15.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-11T05:14:15.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:15.892 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:15.893 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:15.957 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-11T05:14:15.372623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":229687932,"num_objects":56079,"num_object_clones":0,"num_object_copies":112158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13548,"num_objects_unfound":0,"num_objects_dirty":56079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19224,"num_write_kb":76890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56161,"ondisk_log_size":56161,"up":39,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":721628,"kb_used_data":446116,"kb_used_omap":12,"kb_used_meta":275443,"kb_avail":282393892,"statfs":{"total":289910292480,"available":289171345408,"internally_reserved":0,"allocated":456822784,"data_stored":456097727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":282053633},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,4,3,12,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10207626,"num_objects":2487,"num_object_clones":0,"num_object_copies":4974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-26453,"num_objects_unfound":0,"num_objects_dirty":2487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-7446,"num_write_kb":-29749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-8,"num_bytes_recovered":-8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.654672"},"pg_stats":[{"pgid":"2.f","version":"40'3675","reported_seq":3742,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.147344+0000","last_change":"2024-09-11T05:14:12.147344+0000","last_active":"2024-09-11T05:14:12.147344+0000","last_peered":"2024-09-11T05:14:12.143377+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:05.699976+0000","last_became_peered":"2024-09-11T05:14:05.699976+0000","last_unstale":"2024-09-11T05:14:12.147344+0000","last_undegraded":"2024-09-11T05:14:12.147344+0000","last_fullsized":"2024-09-11T05:14:12.147344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3675,"log_dups_size":0,"ondisk_log_size":3675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15052800,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3675,"num_write_kb":14700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'3433","reported_seq":3521,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:10.465402+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:10.465402+0000","last_peered":"2024-09-11T05:14:10.465402+0000","last_clean":"2024-09-11T05:14:10.465402+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:10.465402+0000","last_undegraded":"2024-09-11T05:14:10.465402+0000","last_fullsized":"2024-09-11T05:14:10.465402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3433,"log_dups_size":0,"ondisk_log_size":3433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1831,"num_write_kb":7324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'3390","reported_seq":3497,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.441739+0000","last_change":"2024-09-11T05:14:05.717800+0000","last_active":"2024-09-11T05:14:10.441739+0000","last_peered":"2024-09-11T05:14:10.441739+0000","last_clean":"2024-09-11T05:14:10.441739+0000","last_became_active":"2024-09-11T05:14:05.714077+0000","last_became_peered":"2024-09-11T05:14:05.714077+0000","last_unstale":"2024-09-11T05:14:10.441739+0000","last_undegraded":"2024-09-11T05:14:10.441739+0000","last_fullsized":"2024-09-11T05:14:10.441739+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3390,"log_dups_size":0,"ondisk_log_size":3390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:58.084314+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13869056,"num_objects":3386,"num_object_clones":0,"num_object_copies":6772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":3386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1793,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3386}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'3519","reported_seq":3577,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:10.454140+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:10.454140+0000","last_peered":"2024-09-11T05:14:10.454140+0000","last_clean":"2024-09-11T05:14:10.454140+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:10.454140+0000","last_undegraded":"2024-09-11T05:14:10.454140+0000","last_fullsized":"2024-09-11T05:14:10.454140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3519,"log_dups_size":0,"ondisk_log_size":3519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14413824,"num_objects":3519,"num_object_clones":0,"num_object_copies":7038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3519,"num_write_kb":14076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'3609","reported_seq":3731,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T05:14:12.747020+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:12.747020+0000","last_peered":"2024-09-11T05:14:12.747020+0000","last_clean":"2024-09-11T05:14:12.747020+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:12.747020+0000","last_undegraded":"2024-09-11T05:14:12.747020+0000","last_fullsized":"2024-09-11T05:14:12.747020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":3609,"log_dups_size":0,"ondisk_log_size":3609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753792,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":443,"num_write_kb":1772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'3605","reported_seq":3743,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.750652+0000","last_change":"2024-09-11T05:14:12.151347+0000","last_active":"2024-09-11T05:14:12.151347+0000","last_peered":"2024-09-11T05:14:12.019161+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:05.736779+0000","last_became_peered":"2024-09-11T05:14:05.736779+0000","last_unstale":"2024-09-11T05:14:12.750652+0000","last_undegraded":"2024-09-11T05:14:12.750652+0000","last_fullsized":"2024-09-11T05:14:12.750652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":3605,"log_dups_size":0,"ondisk_log_size":3605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14745600,"num_objects":3600,"num_object_clones":0,"num_object_copies":7200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":149,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'3582","reported_seq":3703,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.745605+0000","last_change":"2024-09-11T05:14:12.145745+0000","last_active":"2024-09-11T05:14:12.145745+0000","last_peered":"2024-09-11T05:14:12.136459+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:05.691077+0000","last_became_peered":"2024-09-11T05:14:05.691077+0000","last_unstale":"2024-09-11T05:14:12.745605+0000","last_undegraded":"2024-09-11T05:14:12.745605+0000","last_fullsized":"2024-09-11T05:14:12.745605+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14647296,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'3513","reported_seq":3630,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T05:14:12.747302+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:12.747302+0000","last_peered":"2024-09-11T05:14:12.747302+0000","last_clean":"2024-09-11T05:14:12.747302+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:12.747302+0000","last_undegraded":"2024-09-11T05:14:12.747302+0000","last_fullsized":"2024-09-11T05:14:12.747302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":3513,"log_dups_size":0,"ondisk_log_size":3513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14381056,"num_objects":3511,"num_object_clones":0,"num_object_copies":7022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.698311+0000","last_change":"2024-09-11T05:14:05.697728+0000","last_active":"2024-09-11T05:14:05.698311+0000","last_peered":"2024-09-11T05:14:05.698311+0000","last_clean":"2024-09-11T05:14:05.698311+0000","last_became_active":"2024-09-11T05:14:05.693462+0000","last_became_peered":"2024-09-11T05:14:05.693462+0000","last_unstale":"2024-09-11T05:14:05.698311+0000","last_undegraded":"2024-09-11T05:14:05.698311+0000","last_fullsized":"2024-09-11T05:14:05.698311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:02.108839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"39'3411","reported_seq":3526,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.446741+0000","last_change":"2024-09-11T05:14:05.721664+0000","last_active":"2024-09-11T05:14:10.446741+0000","last_peered":"2024-09-11T05:14:10.446741+0000","last_clean":"2024-09-11T05:14:10.446741+0000","last_became_active":"2024-09-11T05:14:05.715983+0000","last_became_peered":"2024-09-11T05:14:05.715983+0000","last_unstale":"2024-09-11T05:14:10.446741+0000","last_undegraded":"2024-09-11T05:14:10.446741+0000","last_fullsized":"2024-09-11T05:14:10.446741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3411,"log_dups_size":0,"ondisk_log_size":3411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:02.728889+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3404,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1479,"num_write_kb":5916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:05.693427+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:05.693427+0000","last_peered":"2024-09-11T05:14:05.693427+0000","last_clean":"2024-09-11T05:14:05.693427+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:05.693427+0000","last_undegraded":"2024-09-11T05:14:05.693427+0000","last_fullsized":"2024-09-11T05:14:05.693427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'3450","reported_seq":3604,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.747930+0000","last_change":"2024-09-11T05:14:12.151040+0000","last_active":"2024-09-11T05:14:12.151040+0000","last_peered":"2024-09-11T05:14:12.126790+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:05.731889+0000","last_became_peered":"2024-09-11T05:14:05.731889+0000","last_unstale":"2024-09-11T05:14:12.747930+0000","last_undegraded":"2024-09-11T05:14:12.747930+0000","last_fullsized":"2024-09-11T05:14:12.747930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":3450,"log_dups_size":0,"ondisk_log_size":3450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14098432,"num_objects":3442,"num_object_clones":0,"num_object_copies":6884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":203,"num_write_kb":812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":91,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-11T05:14:12.151989+0000","last_change":"2024-09-11T05:14:12.745117+0000","last_active":"2024-09-11T05:14:12.151989+0000","last_peered":"2024-09-11T05:14:10.724773+0000","last_clean":"2024-09-11T05:14:10.724773+0000","last_became_active":"2024-09-11T05:14:05.711585+0000","last_became_peered":"2024-09-11T05:14:05.711585+0000","last_unstale":"2024-09-11T05:14:12.151989+0000","last_undegraded":"2024-09-11T05:14:12.151989+0000","last_fullsized":"2024-09-11T05:14:12.151989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'3405","reported_seq":3507,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.441360+0000","last_change":"2024-09-11T05:14:05.719410+0000","last_active":"2024-09-11T05:14:10.441360+0000","last_peered":"2024-09-11T05:14:10.441360+0000","last_clean":"2024-09-11T05:14:10.441360+0000","last_became_active":"2024-09-11T05:14:05.713050+0000","last_became_peered":"2024-09-11T05:14:05.713050+0000","last_unstale":"2024-09-11T05:14:10.441360+0000","last_undegraded":"2024-09-11T05:14:10.441360+0000","last_fullsized":"2024-09-11T05:14:10.441360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3405,"log_dups_size":0,"ondisk_log_size":3405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:25.938997+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13926400,"num_objects":3400,"num_object_clones":0,"num_object_copies":6800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3400,"num_objects_unfound":0,"num_objects_dirty":3400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":5736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3400}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":31,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.721143+0000","last_change":"2024-09-11T05:14:05.720914+0000","last_active":"2024-09-11T05:14:05.721143+0000","last_peered":"2024-09-11T05:14:05.721143+0000","last_clean":"2024-09-11T05:14:05.721143+0000","last_became_active":"2024-09-11T05:14:05.715309+0000","last_became_peered":"2024-09-11T05:14:05.715309+0000","last_unstale":"2024-09-11T05:14:05.721143+0000","last_undegraded":"2024-09-11T05:14:05.721143+0000","last_fullsized":"2024-09-11T05:14:05.721143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:40.740249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"39'3363","reported_seq":3466,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.442032+0000","last_change":"2024-09-11T05:14:05.719606+0000","last_active":"2024-09-11T05:14:10.442032+0000","last_peered":"2024-09-11T05:14:10.442032+0000","last_clean":"2024-09-11T05:14:10.442032+0000","last_became_active":"2024-09-11T05:14:05.713184+0000","last_became_peered":"2024-09-11T05:14:05.713184+0000","last_unstale":"2024-09-11T05:14:10.442032+0000","last_undegraded":"2024-09-11T05:14:10.442032+0000","last_fullsized":"2024-09-11T05:14:10.442032+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3363,"log_dups_size":0,"ondisk_log_size":3363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:28:01.133718+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3357,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1266,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'3625","reported_seq":3745,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.749732+0000","last_change":"2024-09-11T05:14:12.149176+0000","last_active":"2024-09-11T05:14:12.149176+0000","last_peered":"2024-09-11T05:14:12.129970+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:05.699635+0000","last_became_peered":"2024-09-11T05:14:05.699635+0000","last_unstale":"2024-09-11T05:14:12.749732+0000","last_undegraded":"2024-09-11T05:14:12.749732+0000","last_fullsized":"2024-09-11T05:14:12.749732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":3625,"log_dups_size":0,"ondisk_log_size":3625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14839808,"num_objects":3623,"num_object_clones":0,"num_object_copies":7246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":658,"num_write_kb":2632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.699979+0000","last_change":"2024-09-11T05:14:05.699588+0000","last_active":"2024-09-11T05:14:05.699979+0000","last_peered":"2024-09-11T05:14:05.699979+0000","last_clean":"2024-09-11T05:14:05.699979+0000","last_became_active":"2024-09-11T05:14:05.698242+0000","last_became_peered":"2024-09-11T05:14:05.698242+0000","last_unstale":"2024-09-11T05:14:05.699979+0000","last_undegraded":"2024-09-11T05:14:05.699979+0000","last_fullsized":"2024-09-11T05:14:05.699979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:51.016428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"40'3503","reported_seq":3625,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.749335+0000","last_change":"2024-09-11T05:14:12.146578+0000","last_active":"2024-09-11T05:14:12.146578+0000","last_peered":"2024-09-11T05:14:12.140466+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:05.700031+0000","last_became_peered":"2024-09-11T05:14:05.700031+0000","last_unstale":"2024-09-11T05:14:12.749335+0000","last_undegraded":"2024-09-11T05:14:12.749335+0000","last_fullsized":"2024-09-11T05:14:12.749335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":3503,"log_dups_size":0,"ondisk_log_size":3503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":369,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:05.694423+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:05.694423+0000","last_peered":"2024-09-11T05:14:05.694423+0000","last_clean":"2024-09-11T05:14:05.694423+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:05.694423+0000","last_undegraded":"2024-09-11T05:14:05.694423+0000","last_fullsized":"2024-09-11T05:14:05.694423+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'3548","reported_seq":3672,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.744251+0000","last_change":"2024-09-11T05:14:12.143940+0000","last_active":"2024-09-11T05:14:12.143940+0000","last_peered":"2024-09-11T05:14:12.119953+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:05.704831+0000","last_became_peered":"2024-09-11T05:14:05.704831+0000","last_unstale":"2024-09-11T05:14:12.744251+0000","last_undegraded":"2024-09-11T05:14:12.744251+0000","last_fullsized":"2024-09-11T05:14:12.744251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14516224,"num_objects":3544,"num_object_clones":0,"num_object_copies":7088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":1224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.705381+0000","last_change":"2024-09-11T05:14:05.704973+0000","last_active":"2024-09-11T05:14:05.705381+0000","last_peered":"2024-09-11T05:14:05.705381+0000","last_clean":"2024-09-11T05:14:05.705381+0000","last_became_active":"2024-09-11T05:14:05.704350+0000","last_became_peered":"2024-09-11T05:14:05.704350+0000","last_unstale":"2024-09-11T05:14:05.705381+0000","last_undegraded":"2024-09-11T05:14:05.705381+0000","last_fullsized":"2024-09-11T05:14:05.705381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:53.268667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":90,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.732163+0000","last_change":"2024-09-11T05:14:05.731809+0000","last_active":"2024-09-11T05:14:05.732163+0000","last_peered":"2024-09-11T05:14:05.732163+0000","last_clean":"2024-09-11T05:14:05.732163+0000","last_became_active":"2024-09-11T05:14:05.730782+0000","last_became_peered":"2024-09-11T05:14:05.730782+0000","last_unstale":"2024-09-11T05:14:05.732163+0000","last_undegraded":"2024-09-11T05:14:05.732163+0000","last_fullsized":"2024-09-11T05:14:05.732163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:12.880663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"40'3527","reported_seq":3659,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.744704+0000","last_change":"2024-09-11T05:14:12.145967+0000","last_active":"2024-09-11T05:14:12.145967+0000","last_peered":"2024-09-11T05:14:12.068296+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:05.733059+0000","last_became_peered":"2024-09-11T05:14:05.733059+0000","last_unstale":"2024-09-11T05:14:12.744704+0000","last_undegraded":"2024-09-11T05:14:12.744704+0000","last_fullsized":"2024-09-11T05:14:12.744704+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":3527,"log_dups_size":0,"ondisk_log_size":3527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14398057,"num_objects":3516,"num_object_clones":0,"num_object_copies":7032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":534,"num_write_kb":2136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":229687913,"num_objects":56077,"num_object_clones":0,"num_object_copies":112154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13547,"num_objects_unfound":0,"num_objects_dirty":56077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19222,"num_write_kb":76888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":457474048,"data_stored":457369678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56158,"ondisk_log_size":56158,"up":28,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340260,"kb_used_data":218324,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94031580,"statfs":{"total":96636764160,"available":96288337920,"internally_reserved":0,"allocated":223563776,"data_stored":223334205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":331776,"data_stored":79659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":32435883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215125,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349348,"kb_used_data":227468,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94022492,"statfs":{"total":96636764160,"available":96279031808,"internally_reserved":0,"allocated":232927232,"data_stored":232683863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124776107},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,3,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232570880,"data_stored":232567401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":224788480,"data_stored":224785001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:15.959 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:16.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 43 20 57928 57908 5.25887 2.48438 0.0290132 0.0148441 2024-09-11T05:14:16.810 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:16.812 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:16.855 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-11T05:14:15.372623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":229687932,"num_objects":56079,"num_object_clones":0,"num_object_copies":112158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13548,"num_objects_unfound":0,"num_objects_dirty":56079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19224,"num_write_kb":76890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56161,"ondisk_log_size":56161,"up":39,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":721628,"kb_used_data":446116,"kb_used_omap":12,"kb_used_meta":275443,"kb_avail":282393892,"statfs":{"total":289910292480,"available":289171345408,"internally_reserved":0,"allocated":456822784,"data_stored":456097727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":282053633},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,4,3,12,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10207626,"num_objects":2487,"num_object_clones":0,"num_object_copies":4974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-26453,"num_objects_unfound":0,"num_objects_dirty":2487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-7446,"num_write_kb":-29749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-8,"num_bytes_recovered":-8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.654672"},"pg_stats":[{"pgid":"2.f","version":"40'3675","reported_seq":3742,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.147344+0000","last_change":"2024-09-11T05:14:12.147344+0000","last_active":"2024-09-11T05:14:12.147344+0000","last_peered":"2024-09-11T05:14:12.143377+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:05.699976+0000","last_became_peered":"2024-09-11T05:14:05.699976+0000","last_unstale":"2024-09-11T05:14:12.147344+0000","last_undegraded":"2024-09-11T05:14:12.147344+0000","last_fullsized":"2024-09-11T05:14:12.147344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3675,"log_dups_size":0,"ondisk_log_size":3675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15052800,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3675,"num_write_kb":14700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'3433","reported_seq":3521,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:10.465402+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:10.465402+0000","last_peered":"2024-09-11T05:14:10.465402+0000","last_clean":"2024-09-11T05:14:10.465402+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:10.465402+0000","last_undegraded":"2024-09-11T05:14:10.465402+0000","last_fullsized":"2024-09-11T05:14:10.465402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3433,"log_dups_size":0,"ondisk_log_size":3433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1831,"num_write_kb":7324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'3390","reported_seq":3497,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.441739+0000","last_change":"2024-09-11T05:14:05.717800+0000","last_active":"2024-09-11T05:14:10.441739+0000","last_peered":"2024-09-11T05:14:10.441739+0000","last_clean":"2024-09-11T05:14:10.441739+0000","last_became_active":"2024-09-11T05:14:05.714077+0000","last_became_peered":"2024-09-11T05:14:05.714077+0000","last_unstale":"2024-09-11T05:14:10.441739+0000","last_undegraded":"2024-09-11T05:14:10.441739+0000","last_fullsized":"2024-09-11T05:14:10.441739+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3390,"log_dups_size":0,"ondisk_log_size":3390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:58.084314+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13869056,"num_objects":3386,"num_object_clones":0,"num_object_copies":6772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":3386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1793,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3386}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'3519","reported_seq":3577,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:10.454140+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:10.454140+0000","last_peered":"2024-09-11T05:14:10.454140+0000","last_clean":"2024-09-11T05:14:10.454140+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:10.454140+0000","last_undegraded":"2024-09-11T05:14:10.454140+0000","last_fullsized":"2024-09-11T05:14:10.454140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3519,"log_dups_size":0,"ondisk_log_size":3519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14413824,"num_objects":3519,"num_object_clones":0,"num_object_copies":7038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3519,"num_write_kb":14076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'3609","reported_seq":3731,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T05:14:12.747020+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:12.747020+0000","last_peered":"2024-09-11T05:14:12.747020+0000","last_clean":"2024-09-11T05:14:12.747020+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:12.747020+0000","last_undegraded":"2024-09-11T05:14:12.747020+0000","last_fullsized":"2024-09-11T05:14:12.747020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":3609,"log_dups_size":0,"ondisk_log_size":3609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753792,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":443,"num_write_kb":1772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'3605","reported_seq":3743,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.750652+0000","last_change":"2024-09-11T05:14:12.151347+0000","last_active":"2024-09-11T05:14:12.151347+0000","last_peered":"2024-09-11T05:14:12.019161+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:05.736779+0000","last_became_peered":"2024-09-11T05:14:05.736779+0000","last_unstale":"2024-09-11T05:14:12.750652+0000","last_undegraded":"2024-09-11T05:14:12.750652+0000","last_fullsized":"2024-09-11T05:14:12.750652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":3605,"log_dups_size":0,"ondisk_log_size":3605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14745600,"num_objects":3600,"num_object_clones":0,"num_object_copies":7200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":149,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'3582","reported_seq":3703,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.745605+0000","last_change":"2024-09-11T05:14:12.145745+0000","last_active":"2024-09-11T05:14:12.145745+0000","last_peered":"2024-09-11T05:14:12.136459+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:05.691077+0000","last_became_peered":"2024-09-11T05:14:05.691077+0000","last_unstale":"2024-09-11T05:14:12.745605+0000","last_undegraded":"2024-09-11T05:14:12.745605+0000","last_fullsized":"2024-09-11T05:14:12.745605+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14647296,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'3513","reported_seq":3630,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T05:14:12.747302+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:12.747302+0000","last_peered":"2024-09-11T05:14:12.747302+0000","last_clean":"2024-09-11T05:14:12.747302+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:12.747302+0000","last_undegraded":"2024-09-11T05:14:12.747302+0000","last_fullsized":"2024-09-11T05:14:12.747302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":3513,"log_dups_size":0,"ondisk_log_size":3513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14381056,"num_objects":3511,"num_object_clones":0,"num_object_copies":7022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.698311+0000","last_change":"2024-09-11T05:14:05.697728+0000","last_active":"2024-09-11T05:14:05.698311+0000","last_peered":"2024-09-11T05:14:05.698311+0000","last_clean":"2024-09-11T05:14:05.698311+0000","last_became_active":"2024-09-11T05:14:05.693462+0000","last_became_peered":"2024-09-11T05:14:05.693462+0000","last_unstale":"2024-09-11T05:14:05.698311+0000","last_undegraded":"2024-09-11T05:14:05.698311+0000","last_fullsized":"2024-09-11T05:14:05.698311+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:02.108839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"39'3411","reported_seq":3526,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.446741+0000","last_change":"2024-09-11T05:14:05.721664+0000","last_active":"2024-09-11T05:14:10.446741+0000","last_peered":"2024-09-11T05:14:10.446741+0000","last_clean":"2024-09-11T05:14:10.446741+0000","last_became_active":"2024-09-11T05:14:05.715983+0000","last_became_peered":"2024-09-11T05:14:05.715983+0000","last_unstale":"2024-09-11T05:14:10.446741+0000","last_undegraded":"2024-09-11T05:14:10.446741+0000","last_fullsized":"2024-09-11T05:14:10.446741+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3411,"log_dups_size":0,"ondisk_log_size":3411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:02.728889+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3404,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1479,"num_write_kb":5916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:05.693427+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:05.693427+0000","last_peered":"2024-09-11T05:14:05.693427+0000","last_clean":"2024-09-11T05:14:05.693427+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:05.693427+0000","last_undegraded":"2024-09-11T05:14:05.693427+0000","last_fullsized":"2024-09-11T05:14:05.693427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'3450","reported_seq":3604,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.747930+0000","last_change":"2024-09-11T05:14:12.151040+0000","last_active":"2024-09-11T05:14:12.151040+0000","last_peered":"2024-09-11T05:14:12.126790+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:05.731889+0000","last_became_peered":"2024-09-11T05:14:05.731889+0000","last_unstale":"2024-09-11T05:14:12.747930+0000","last_undegraded":"2024-09-11T05:14:12.747930+0000","last_fullsized":"2024-09-11T05:14:12.747930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":3450,"log_dups_size":0,"ondisk_log_size":3450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14098432,"num_objects":3442,"num_object_clones":0,"num_object_copies":6884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":203,"num_write_kb":812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":91,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-11T05:14:12.151989+0000","last_change":"2024-09-11T05:14:12.745117+0000","last_active":"2024-09-11T05:14:12.151989+0000","last_peered":"2024-09-11T05:14:10.724773+0000","last_clean":"2024-09-11T05:14:10.724773+0000","last_became_active":"2024-09-11T05:14:05.711585+0000","last_became_peered":"2024-09-11T05:14:05.711585+0000","last_unstale":"2024-09-11T05:14:12.151989+0000","last_undegraded":"2024-09-11T05:14:12.151989+0000","last_fullsized":"2024-09-11T05:14:12.151989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'3405","reported_seq":3507,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.441360+0000","last_change":"2024-09-11T05:14:05.719410+0000","last_active":"2024-09-11T05:14:10.441360+0000","last_peered":"2024-09-11T05:14:10.441360+0000","last_clean":"2024-09-11T05:14:10.441360+0000","last_became_active":"2024-09-11T05:14:05.713050+0000","last_became_peered":"2024-09-11T05:14:05.713050+0000","last_unstale":"2024-09-11T05:14:10.441360+0000","last_undegraded":"2024-09-11T05:14:10.441360+0000","last_fullsized":"2024-09-11T05:14:10.441360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3405,"log_dups_size":0,"ondisk_log_size":3405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:25.938997+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13926400,"num_objects":3400,"num_object_clones":0,"num_object_copies":6800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3400,"num_objects_unfound":0,"num_objects_dirty":3400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":5736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3400}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":31,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.721143+0000","last_change":"2024-09-11T05:14:05.720914+0000","last_active":"2024-09-11T05:14:05.721143+0000","last_peered":"2024-09-11T05:14:05.721143+0000","last_clean":"2024-09-11T05:14:05.721143+0000","last_became_active":"2024-09-11T05:14:05.715309+0000","last_became_peered":"2024-09-11T05:14:05.715309+0000","last_unstale":"2024-09-11T05:14:05.721143+0000","last_undegraded":"2024-09-11T05:14:05.721143+0000","last_fullsized":"2024-09-11T05:14:05.721143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:40.740249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"39'3363","reported_seq":3466,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:10.442032+0000","last_change":"2024-09-11T05:14:05.719606+0000","last_active":"2024-09-11T05:14:10.442032+0000","last_peered":"2024-09-11T05:14:10.442032+0000","last_clean":"2024-09-11T05:14:10.442032+0000","last_became_active":"2024-09-11T05:14:05.713184+0000","last_became_peered":"2024-09-11T05:14:05.713184+0000","last_unstale":"2024-09-11T05:14:10.442032+0000","last_undegraded":"2024-09-11T05:14:10.442032+0000","last_fullsized":"2024-09-11T05:14:10.442032+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3363,"log_dups_size":0,"ondisk_log_size":3363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:28:01.133718+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3357,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1266,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'3625","reported_seq":3745,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.749732+0000","last_change":"2024-09-11T05:14:12.149176+0000","last_active":"2024-09-11T05:14:12.149176+0000","last_peered":"2024-09-11T05:14:12.129970+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:05.699635+0000","last_became_peered":"2024-09-11T05:14:05.699635+0000","last_unstale":"2024-09-11T05:14:12.749732+0000","last_undegraded":"2024-09-11T05:14:12.749732+0000","last_fullsized":"2024-09-11T05:14:12.749732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":3625,"log_dups_size":0,"ondisk_log_size":3625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14839808,"num_objects":3623,"num_object_clones":0,"num_object_copies":7246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":658,"num_write_kb":2632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.699979+0000","last_change":"2024-09-11T05:14:05.699588+0000","last_active":"2024-09-11T05:14:05.699979+0000","last_peered":"2024-09-11T05:14:05.699979+0000","last_clean":"2024-09-11T05:14:05.699979+0000","last_became_active":"2024-09-11T05:14:05.698242+0000","last_became_peered":"2024-09-11T05:14:05.698242+0000","last_unstale":"2024-09-11T05:14:05.699979+0000","last_undegraded":"2024-09-11T05:14:05.699979+0000","last_fullsized":"2024-09-11T05:14:05.699979+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:51.016428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"40'3503","reported_seq":3625,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.749335+0000","last_change":"2024-09-11T05:14:12.146578+0000","last_active":"2024-09-11T05:14:12.146578+0000","last_peered":"2024-09-11T05:14:12.140466+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:05.700031+0000","last_became_peered":"2024-09-11T05:14:05.700031+0000","last_unstale":"2024-09-11T05:14:12.749335+0000","last_undegraded":"2024-09-11T05:14:12.749335+0000","last_fullsized":"2024-09-11T05:14:12.749335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":3503,"log_dups_size":0,"ondisk_log_size":3503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":369,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T05:14:05.694423+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:05.694423+0000","last_peered":"2024-09-11T05:14:05.694423+0000","last_clean":"2024-09-11T05:14:05.694423+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:05.694423+0000","last_undegraded":"2024-09-11T05:14:05.694423+0000","last_fullsized":"2024-09-11T05:14:05.694423+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'3548","reported_seq":3672,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.744251+0000","last_change":"2024-09-11T05:14:12.143940+0000","last_active":"2024-09-11T05:14:12.143940+0000","last_peered":"2024-09-11T05:14:12.119953+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:05.704831+0000","last_became_peered":"2024-09-11T05:14:05.704831+0000","last_unstale":"2024-09-11T05:14:12.744251+0000","last_undegraded":"2024-09-11T05:14:12.744251+0000","last_fullsized":"2024-09-11T05:14:12.744251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14516224,"num_objects":3544,"num_object_clones":0,"num_object_copies":7088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":1224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.705381+0000","last_change":"2024-09-11T05:14:05.704973+0000","last_active":"2024-09-11T05:14:05.705381+0000","last_peered":"2024-09-11T05:14:05.705381+0000","last_clean":"2024-09-11T05:14:05.705381+0000","last_became_active":"2024-09-11T05:14:05.704350+0000","last_became_peered":"2024-09-11T05:14:05.704350+0000","last_unstale":"2024-09-11T05:14:05.705381+0000","last_undegraded":"2024-09-11T05:14:05.705381+0000","last_fullsized":"2024-09-11T05:14:05.705381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:53.268667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":90,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T05:14:05.732163+0000","last_change":"2024-09-11T05:14:05.731809+0000","last_active":"2024-09-11T05:14:05.732163+0000","last_peered":"2024-09-11T05:14:05.732163+0000","last_clean":"2024-09-11T05:14:05.732163+0000","last_became_active":"2024-09-11T05:14:05.730782+0000","last_became_peered":"2024-09-11T05:14:05.730782+0000","last_unstale":"2024-09-11T05:14:05.732163+0000","last_undegraded":"2024-09-11T05:14:05.732163+0000","last_fullsized":"2024-09-11T05:14:05.732163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:12.880663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"40'3527","reported_seq":3659,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.744704+0000","last_change":"2024-09-11T05:14:12.145967+0000","last_active":"2024-09-11T05:14:12.145967+0000","last_peered":"2024-09-11T05:14:12.068296+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:05.733059+0000","last_became_peered":"2024-09-11T05:14:05.733059+0000","last_unstale":"2024-09-11T05:14:12.744704+0000","last_undegraded":"2024-09-11T05:14:12.744704+0000","last_fullsized":"2024-09-11T05:14:12.744704+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":3527,"log_dups_size":0,"ondisk_log_size":3527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14398057,"num_objects":3516,"num_object_clones":0,"num_object_copies":7032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":534,"num_write_kb":2136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":229687913,"num_objects":56077,"num_object_clones":0,"num_object_copies":112154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13547,"num_objects_unfound":0,"num_objects_dirty":56077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19222,"num_write_kb":76888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":457474048,"data_stored":457369678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56158,"ondisk_log_size":56158,"up":28,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340260,"kb_used_data":218324,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94031580,"statfs":{"total":96636764160,"available":96288337920,"internally_reserved":0,"allocated":223563776,"data_stored":223334205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":331776,"data_stored":79659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":32435883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215125,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349348,"kb_used_data":227468,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94022492,"statfs":{"total":96636764160,"available":96279031808,"internally_reserved":0,"allocated":232927232,"data_stored":232683863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124776107},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,3,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232570880,"data_stored":232567401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":224788480,"data_stored":224785001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:16.856 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:17.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 44 20 58566 58546 5.1959 2.49219 0.0531341 0.0150269 2024-09-11T05:14:17.777 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:17.778 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:17.822 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-11T05:14:17.372957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233751164,"num_objects":57071,"num_object_clones":0,"num_object_copies":114142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10546,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":57071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20216,"num_write_kb":80858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":106515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57153,"ondisk_log_size":57153,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":759440,"kb_used_data":457556,"kb_used_omap":12,"kb_used_meta":301811,"kb_avail":282356080,"statfs":{"total":289910292480,"available":289132625920,"internally_reserved":0,"allocated":468537344,"data_stored":467724442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":309054465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,6,3,8,6,0,0,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14270858,"num_objects":3479,"num_object_clones":0,"num_object_copies":6958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10546,"num_objects_misplaced":-36548,"num_objects_unfound":0,"num_objects_dirty":3479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-6454,"num_write_kb":-25781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":94208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001166"},"pg_stats":[{"pgid":"2.f","version":"40'3675","reported_seq":3742,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.147344+0000","last_change":"2024-09-11T05:14:12.147344+0000","last_active":"2024-09-11T05:14:12.147344+0000","last_peered":"2024-09-11T05:14:12.143377+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:05.699976+0000","last_became_peered":"2024-09-11T05:14:05.699976+0000","last_unstale":"2024-09-11T05:14:12.147344+0000","last_undegraded":"2024-09-11T05:14:12.147344+0000","last_fullsized":"2024-09-11T05:14:12.147344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3675,"log_dups_size":0,"ondisk_log_size":3675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15052800,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3675,"num_write_kb":14700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3594","reported_seq":3691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:15.398618+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:15.398618+0000","last_peered":"2024-09-11T05:14:15.398618+0000","last_clean":"2024-09-11T05:14:15.398618+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:15.398618+0000","last_undegraded":"2024-09-11T05:14:15.398618+0000","last_fullsized":"2024-09-11T05:14:15.398618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3594,"log_dups_size":0,"ondisk_log_size":3594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1992,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3550","reported_seq":3670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:15.427333+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:15.427333+0000","last_peered":"2024-09-11T05:14:15.427333+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:15.427333+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3550,"log_dups_size":0,"ondisk_log_size":3550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1953,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":52}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3676","reported_seq":3742,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:15.403572+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:15.403572+0000","last_peered":"2024-09-11T05:14:15.403572+0000","last_clean":"2024-09-11T05:14:15.403572+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:15.403572+0000","last_undegraded":"2024-09-11T05:14:15.403572+0000","last_fullsized":"2024-09-11T05:14:15.403572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15056896,"num_objects":3676,"num_object_clones":0,"num_object_copies":7352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3676,"num_write_kb":14704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'3609","reported_seq":3731,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T05:14:12.747020+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:12.747020+0000","last_peered":"2024-09-11T05:14:12.747020+0000","last_clean":"2024-09-11T05:14:12.747020+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:12.747020+0000","last_undegraded":"2024-09-11T05:14:12.747020+0000","last_fullsized":"2024-09-11T05:14:12.747020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":3609,"log_dups_size":0,"ondisk_log_size":3609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753792,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":443,"num_write_kb":1772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'3605","reported_seq":3743,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.750652+0000","last_change":"2024-09-11T05:14:12.151347+0000","last_active":"2024-09-11T05:14:12.151347+0000","last_peered":"2024-09-11T05:14:12.019161+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:05.736779+0000","last_became_peered":"2024-09-11T05:14:05.736779+0000","last_unstale":"2024-09-11T05:14:12.750652+0000","last_undegraded":"2024-09-11T05:14:12.750652+0000","last_fullsized":"2024-09-11T05:14:12.750652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":3605,"log_dups_size":0,"ondisk_log_size":3605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14745600,"num_objects":3600,"num_object_clones":0,"num_object_copies":7200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":149,"num_write_kb":596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'3582","reported_seq":3703,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.745605+0000","last_change":"2024-09-11T05:14:12.145745+0000","last_active":"2024-09-11T05:14:12.145745+0000","last_peered":"2024-09-11T05:14:12.136459+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:05.691077+0000","last_became_peered":"2024-09-11T05:14:05.691077+0000","last_unstale":"2024-09-11T05:14:12.745605+0000","last_undegraded":"2024-09-11T05:14:12.745605+0000","last_fullsized":"2024-09-11T05:14:12.745605+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14647296,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'3513","reported_seq":3630,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T05:14:12.747302+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:12.747302+0000","last_peered":"2024-09-11T05:14:12.747302+0000","last_clean":"2024-09-11T05:14:12.747302+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:12.747302+0000","last_undegraded":"2024-09-11T05:14:12.747302+0000","last_fullsized":"2024-09-11T05:14:12.747302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":3513,"log_dups_size":0,"ondisk_log_size":3513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14381056,"num_objects":3511,"num_object_clones":0,"num_object_copies":7022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3582","reported_seq":3708,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:15.460646+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:15.460646+0000","last_peered":"2024-09-11T05:14:15.460646+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:15.460646+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14643200,"num_objects":3575,"num_object_clones":0,"num_object_copies":7150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1650,"num_write_kb":6600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":50}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'3450","reported_seq":3604,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.747930+0000","last_change":"2024-09-11T05:14:12.151040+0000","last_active":"2024-09-11T05:14:12.151040+0000","last_peered":"2024-09-11T05:14:12.126790+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:05.731889+0000","last_became_peered":"2024-09-11T05:14:05.731889+0000","last_unstale":"2024-09-11T05:14:12.747930+0000","last_undegraded":"2024-09-11T05:14:12.747930+0000","last_fullsized":"2024-09-11T05:14:12.747930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":3450,"log_dups_size":0,"ondisk_log_size":3450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14098432,"num_objects":3442,"num_object_clones":0,"num_object_copies":6884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":203,"num_write_kb":812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":91,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-11T05:14:12.151989+0000","last_change":"2024-09-11T05:14:12.745117+0000","last_active":"2024-09-11T05:14:12.151989+0000","last_peered":"2024-09-11T05:14:10.724773+0000","last_clean":"2024-09-11T05:14:10.724773+0000","last_became_active":"2024-09-11T05:14:05.711585+0000","last_became_peered":"2024-09-11T05:14:05.711585+0000","last_unstale":"2024-09-11T05:14:12.151989+0000","last_undegraded":"2024-09-11T05:14:12.151989+0000","last_fullsized":"2024-09-11T05:14:12.151989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3575","reported_seq":3687,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:15.406401+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:15.406401+0000","last_peered":"2024-09-11T05:14:15.406401+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:15.406401+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14622720,"num_objects":3570,"num_object_clones":0,"num_object_copies":7140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":6416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":48}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3536","reported_seq":3698,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:15.466220+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:15.466220+0000","last_peered":"2024-09-11T05:14:15.466220+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:15.466220+0000","last_undegraded":"2024-09-11T05:14:15.466220+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14458880,"num_objects":3530,"num_object_clones":0,"num_object_copies":7060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":3530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1439,"num_write_kb":5756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":102400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3453},{"shards":"2","objects":77}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'3625","reported_seq":3745,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.749732+0000","last_change":"2024-09-11T05:14:12.149176+0000","last_active":"2024-09-11T05:14:12.149176+0000","last_peered":"2024-09-11T05:14:12.129970+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:05.699635+0000","last_became_peered":"2024-09-11T05:14:05.699635+0000","last_unstale":"2024-09-11T05:14:12.749732+0000","last_undegraded":"2024-09-11T05:14:12.749732+0000","last_fullsized":"2024-09-11T05:14:12.749732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":3625,"log_dups_size":0,"ondisk_log_size":3625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14839808,"num_objects":3623,"num_object_clones":0,"num_object_copies":7246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":658,"num_write_kb":2632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"40'3503","reported_seq":3625,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.749335+0000","last_change":"2024-09-11T05:14:12.146578+0000","last_active":"2024-09-11T05:14:12.146578+0000","last_peered":"2024-09-11T05:14:12.140466+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:05.700031+0000","last_became_peered":"2024-09-11T05:14:05.700031+0000","last_unstale":"2024-09-11T05:14:12.749335+0000","last_undegraded":"2024-09-11T05:14:12.749335+0000","last_fullsized":"2024-09-11T05:14:12.749335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":3503,"log_dups_size":0,"ondisk_log_size":3503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":369,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'3548","reported_seq":3672,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.744251+0000","last_change":"2024-09-11T05:14:12.143940+0000","last_active":"2024-09-11T05:14:12.143940+0000","last_peered":"2024-09-11T05:14:12.119953+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:05.704831+0000","last_became_peered":"2024-09-11T05:14:05.704831+0000","last_unstale":"2024-09-11T05:14:12.744251+0000","last_undegraded":"2024-09-11T05:14:12.744251+0000","last_fullsized":"2024-09-11T05:14:12.744251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14516224,"num_objects":3544,"num_object_clones":0,"num_object_copies":7088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":1224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"40'3527","reported_seq":3659,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T05:14:12.744704+0000","last_change":"2024-09-11T05:14:12.145967+0000","last_active":"2024-09-11T05:14:12.145967+0000","last_peered":"2024-09-11T05:14:12.068296+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:05.733059+0000","last_became_peered":"2024-09-11T05:14:05.733059+0000","last_unstale":"2024-09-11T05:14:12.744704+0000","last_undegraded":"2024-09-11T05:14:12.744704+0000","last_fullsized":"2024-09-11T05:14:12.744704+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":3527,"log_dups_size":0,"ondisk_log_size":3527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14398057,"num_objects":3516,"num_object_clones":0,"num_object_copies":7032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":534,"num_write_kb":2136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233751145,"num_objects":57069,"num_object_clones":0,"num_object_copies":114138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10546,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":57069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20214,"num_write_kb":80856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":106496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":469786624,"data_stored":469699649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57150,"ondisk_log_size":57150,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358988,"kb_used_data":228732,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94012852,"statfs":{"total":96636764160,"available":96269160448,"internally_reserved":0,"allocated":234221568,"data_stored":233977109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51104,"kb_used_data":1356,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94320736,"statfs":{"total":96636764160,"available":96584433664,"internally_reserved":0,"allocated":1388544,"data_stored":1063470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":50917035},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215125,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349348,"kb_used_data":227468,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94022492,"statfs":{"total":96636764160,"available":96279031808,"internally_reserved":0,"allocated":232927232,"data_stored":232683863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124776107},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,3,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232570880,"data_stored":232567401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2998272,"data_stored":2918255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234217472,"data_stored":234213993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:17.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:14:17.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "40'3675", 'reported_seq': 3742, 'reported_epoch': 41, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.147344+0000', 'last_change': '2024-09-11T05:14:12.147344+0000', 'last_active': '2024-09-11T05:14:12.147344+0000', 'last_peered': '2024-09-11T05:14:12.143377+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:05.699976+0000', 'last_became_peered': '2024-09-11T05:14:05.699976+0000', 'last_unstale': '2024-09-11T05:14:12.147344+0000', 'last_undegraded': '2024-09-11T05:14:12.147344+0000', 'last_fullsized': '2024-09-11T05:14:12.147344+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 3675, 'log_dups_size': 0, 'ondisk_log_size': 3675, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15052800, 'num_objects': 3675, 'num_object_clones': 0, 'num_object_copies': 7350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3675, 'num_write_kb': 14700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:14:17.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'3550", 'reported_seq': 3670, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:15.427333+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:14:15.427333+0000', 'last_peered': '2024-09-11T05:14:15.427333+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:14:15.427333+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 3550, 'log_dups_size': 0, 'ondisk_log_size': 3550, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14524416, 'num_objects': 3546, 'num_object_clones': 0, 'num_object_copies': 7092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1953, 'num_write_kb': 7812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 52}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:17.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:14:17.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "40'3605", 'reported_seq': 3743, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.750652+0000', 'last_change': '2024-09-11T05:14:12.151347+0000', 'last_active': '2024-09-11T05:14:12.151347+0000', 'last_peered': '2024-09-11T05:14:12.019161+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:05.736779+0000', 'last_became_peered': '2024-09-11T05:14:05.736779+0000', 'last_unstale': '2024-09-11T05:14:12.750652+0000', 'last_undegraded': '2024-09-11T05:14:12.750652+0000', 'last_fullsized': '2024-09-11T05:14:12.750652+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 3605, 'log_dups_size': 0, 'ondisk_log_size': 3605, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14745600, 'num_objects': 3600, 'num_object_clones': 0, 'num_object_copies': 7200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 149, 'num_write_kb': 596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:14:17.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "40'3582", 'reported_seq': 3703, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.745605+0000', 'last_change': '2024-09-11T05:14:12.145745+0000', 'last_active': '2024-09-11T05:14:12.145745+0000', 'last_peered': '2024-09-11T05:14:12.136459+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:05.691077+0000', 'last_became_peered': '2024-09-11T05:14:05.691077+0000', 'last_unstale': '2024-09-11T05:14:12.745605+0000', 'last_undegraded': '2024-09-11T05:14:12.745605+0000', 'last_fullsized': '2024-09-11T05:14:12.745605+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 3582, 'log_dups_size': 0, 'ondisk_log_size': 3582, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14647296, 'num_objects': 3576, 'num_object_clones': 0, 'num_object_copies': 7152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 747, 'num_write_kb': 2988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:14:17.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'3582", 'reported_seq': 3708, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:15.460646+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:14:15.460646+0000', 'last_peered': '2024-09-11T05:14:15.460646+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:14:15.460646+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 3582, 'log_dups_size': 0, 'ondisk_log_size': 3582, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14643200, 'num_objects': 3575, 'num_object_clones': 0, 'num_object_copies': 7150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1650, 'num_write_kb': 6600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 50}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:17.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:14:17.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "40'3450", 'reported_seq': 3604, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.747930+0000', 'last_change': '2024-09-11T05:14:12.151040+0000', 'last_active': '2024-09-11T05:14:12.151040+0000', 'last_peered': '2024-09-11T05:14:12.126790+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:05.731889+0000', 'last_became_peered': '2024-09-11T05:14:05.731889+0000', 'last_unstale': '2024-09-11T05:14:12.747930+0000', 'last_undegraded': '2024-09-11T05:14:12.747930+0000', 'last_fullsized': '2024-09-11T05:14:12.747930+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 3450, 'log_dups_size': 0, 'ondisk_log_size': 3450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14098432, 'num_objects': 3442, 'num_object_clones': 0, 'num_object_copies': 6884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 203, 'num_write_kb': 812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.2 is not active+clean 2024-09-11T05:14:17.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 91, 'reported_epoch': 41, 'state': 'peering', 'last_fresh': '2024-09-11T05:14:12.151989+0000', 'last_change': '2024-09-11T05:14:12.745117+0000', 'last_active': '2024-09-11T05:14:12.151989+0000', 'last_peered': '2024-09-11T05:14:10.724773+0000', 'last_clean': '2024-09-11T05:14:10.724773+0000', 'last_became_active': '2024-09-11T05:14:05.711585+0000', 'last_became_peered': '2024-09-11T05:14:05.711585+0000', 'last_unstale': '2024-09-11T05:14:12.151989+0000', 'last_undegraded': '2024-09-11T05:14:12.151989+0000', 'last_fullsized': '2024-09-11T05:14:12.151989+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:18.438492+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:18.438492+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:18.438492+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.001247845, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'3575", 'reported_seq': 3687, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:15.406401+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:14:15.406401+0000', 'last_peered': '2024-09-11T05:14:15.406401+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:14:15.406401+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 3575, 'log_dups_size': 0, 'ondisk_log_size': 3575, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14622720, 'num_objects': 3570, 'num_object_clones': 0, 'num_object_copies': 7140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1604, 'num_write_kb': 6416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 48}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'3536", 'reported_seq': 3698, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:14:15.466220+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:14:15.466220+0000', 'last_peered': '2024-09-11T05:14:15.466220+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:14:15.466220+0000', 'last_undegraded': '2024-09-11T05:14:15.466220+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 3536, 'log_dups_size': 0, 'ondisk_log_size': 3536, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14458880, 'num_objects': 3530, 'num_object_clones': 0, 'num_object_copies': 7060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3453, 'num_objects_unfound': 0, 'num_objects_dirty': 3530, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1439, 'num_write_kb': 5756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 102400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3453}, {'shards': '2', 'objects': 77}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "40'3625", 'reported_seq': 3745, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.749732+0000', 'last_change': '2024-09-11T05:14:12.149176+0000', 'last_active': '2024-09-11T05:14:12.149176+0000', 'last_peered': '2024-09-11T05:14:12.129970+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:05.699635+0000', 'last_became_peered': '2024-09-11T05:14:05.699635+0000', 'last_unstale': '2024-09-11T05:14:12.749732+0000', 'last_undegraded': '2024-09-11T05:14:12.749732+0000', 'last_fullsized': '2024-09-11T05:14:12.749732+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 3625, 'log_dups_size': 0, 'ondisk_log_size': 3625, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14839808, 'num_objects': 3623, 'num_object_clones': 0, 'num_object_copies': 7246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 658, 'num_write_kb': 2632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "40'3503", 'reported_seq': 3625, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.749335+0000', 'last_change': '2024-09-11T05:14:12.146578+0000', 'last_active': '2024-09-11T05:14:12.146578+0000', 'last_peered': '2024-09-11T05:14:12.140466+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:05.700031+0000', 'last_became_peered': '2024-09-11T05:14:05.700031+0000', 'last_unstale': '2024-09-11T05:14:12.749335+0000', 'last_undegraded': '2024-09-11T05:14:12.749335+0000', 'last_fullsized': '2024-09-11T05:14:12.749335+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 3503, 'log_dups_size': 0, 'ondisk_log_size': 3503, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14315520, 'num_objects': 3495, 'num_object_clones': 0, 'num_object_copies': 6990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 369, 'num_write_kb': 1476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:14:17.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "40'3548", 'reported_seq': 3672, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.744251+0000', 'last_change': '2024-09-11T05:14:12.143940+0000', 'last_active': '2024-09-11T05:14:12.143940+0000', 'last_peered': '2024-09-11T05:14:12.119953+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:05.704831+0000', 'last_became_peered': '2024-09-11T05:14:05.704831+0000', 'last_unstale': '2024-09-11T05:14:12.744251+0000', 'last_undegraded': '2024-09-11T05:14:12.744251+0000', 'last_fullsized': '2024-09-11T05:14:12.744251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 3548, 'log_dups_size': 0, 'ondisk_log_size': 3548, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14516224, 'num_objects': 3544, 'num_object_clones': 0, 'num_object_copies': 7088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 306, 'num_write_kb': 1224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:14:17.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "40'3527", 'reported_seq': 3659, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T05:14:12.744704+0000', 'last_change': '2024-09-11T05:14:12.145967+0000', 'last_active': '2024-09-11T05:14:12.145967+0000', 'last_peered': '2024-09-11T05:14:12.068296+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:05.733059+0000', 'last_became_peered': '2024-09-11T05:14:05.733059+0000', 'last_unstale': '2024-09-11T05:14:12.744704+0000', 'last_undegraded': '2024-09-11T05:14:12.744704+0000', 'last_fullsized': '2024-09-11T05:14:12.744704+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 3527, 'log_dups_size': 0, 'ondisk_log_size': 3527, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14398057, 'num_objects': 3516, 'num_object_clones': 0, 'num_object_copies': 7032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 534, 'num_write_kb': 2136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:17.827 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:14:18.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 45 20 59174 59154 5.13322 2.375 0.0298883 0.015208 2024-09-11T05:14:18.805 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:18.805 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10546/114142 objects degraded (9.239%), 3 pgs degraded","count":3},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":136,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":11},{"state_name":"remapped+peering","count":8},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":57071,"data_bytes":233751164,"bytes_used":777666560,"bytes_avail":289132625920,"bytes_total":289910292480,"inactive_pgs_ratio":0.375,"degraded_objects":10546,"degraded_total":114142,"degraded_ratio":0.092393685058961644,"misplaced_objects":3453,"misplaced_total":114142,"misplaced_ratio":0.030251791627972174,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":15698,"recovering_keys_per_sec":0,"num_objects_recovered":17,"num_bytes_recovered":94208,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:14:18.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:14:18.860 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:19.081 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 46 20 59797 59777 5.07428 2.43359 0.0825584 0.0153859 2024-09-11T05:14:19.685 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:19.687 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:19.798 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-11T05:14:19.373432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":240689788,"num_objects":58765,"num_object_clones":0,"num_object_copies":117530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":58765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21910,"num_write_kb":87634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":106515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58847,"ondisk_log_size":58847,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":793100,"kb_used_data":464856,"kb_used_omap":12,"kb_used_meta":328179,"kb_avail":282322420,"statfs":{"total":289910292480,"available":289098158080,"internally_reserved":0,"allocated":476012544,"data_stored":475194475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":336055297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,11,1,5,6,4,1,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21209482,"num_objects":5173,"num_object_clones":0,"num_object_copies":10346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":-36548,"num_objects_unfound":0,"num_objects_dirty":5173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-4760,"num_write_kb":-19005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":94208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.651591"},"pg_stats":[{"pgid":"2.f","version":"43'3842","reported_seq":3915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582082+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:18.582082+0000","last_peered":"2024-09-11T05:14:18.582082+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:18.582082+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3842,"log_dups_size":0,"ondisk_log_size":3842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15736832,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3842,"num_write_kb":15368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3594","reported_seq":3691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:15.398618+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:15.398618+0000","last_peered":"2024-09-11T05:14:15.398618+0000","last_clean":"2024-09-11T05:14:15.398618+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:15.398618+0000","last_undegraded":"2024-09-11T05:14:15.398618+0000","last_fullsized":"2024-09-11T05:14:15.398618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3594,"log_dups_size":0,"ondisk_log_size":3594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1992,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3550","reported_seq":3670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:15.427333+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:15.427333+0000","last_peered":"2024-09-11T05:14:15.427333+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:15.427333+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3550,"log_dups_size":0,"ondisk_log_size":3550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1953,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":52}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3676","reported_seq":3742,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:15.403572+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:15.403572+0000","last_peered":"2024-09-11T05:14:15.403572+0000","last_clean":"2024-09-11T05:14:15.403572+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:15.403572+0000","last_undegraded":"2024-09-11T05:14:15.403572+0000","last_fullsized":"2024-09-11T05:14:15.403572+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15056896,"num_objects":3676,"num_object_clones":0,"num_object_copies":7352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3676,"num_write_kb":14704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3783","reported_seq":3907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:18.552413+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:18.552413+0000","last_peered":"2024-09-11T05:14:18.552413+0000","last_clean":"2024-09-11T05:14:18.552413+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:18.552413+0000","last_undegraded":"2024-09-11T05:14:18.552413+0000","last_fullsized":"2024-09-11T05:14:18.552413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15466496,"num_objects":3776,"num_object_clones":0,"num_object_copies":7552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":2468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3782","reported_seq":3924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.560308+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:18.560308+0000","last_peered":"2024-09-11T05:14:18.560308+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:18.560308+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":3782,"log_dups_size":0,"ondisk_log_size":3782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":326,"num_write_kb":1304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":177},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3756","reported_seq":3883,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.524540+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:18.524540+0000","last_peered":"2024-09-11T05:14:18.524540+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:18.524540+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":3756,"log_dups_size":0,"ondisk_log_size":3756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360000,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":3684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3677","reported_seq":3796,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:18.546577+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:18.546577+0000","last_peered":"2024-09-11T05:14:18.546577+0000","last_clean":"2024-09-11T05:14:18.546577+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:18.546577+0000","last_undegraded":"2024-09-11T05:14:18.546577+0000","last_fullsized":"2024-09-11T05:14:18.546577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":3677,"log_dups_size":0,"ondisk_log_size":3677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15052800,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":980,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3582","reported_seq":3708,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:15.460646+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:15.460646+0000","last_peered":"2024-09-11T05:14:15.460646+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:15.460646+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14643200,"num_objects":3575,"num_object_clones":0,"num_object_copies":7150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1650,"num_write_kb":6600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":50}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3623","reported_seq":3780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.572690+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:18.572690+0000","last_peered":"2024-09-11T05:14:18.572690+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:18.572690+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":3623,"log_dups_size":0,"ondisk_log_size":3623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14807040,"num_objects":3615,"num_object_clones":0,"num_object_copies":7230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":173},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3575","reported_seq":3687,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:15.406401+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:15.406401+0000","last_peered":"2024-09-11T05:14:15.406401+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:15.406401+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14622720,"num_objects":3570,"num_object_clones":0,"num_object_copies":7140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1604,"num_write_kb":6416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":48}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3536","reported_seq":3698,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:15.466220+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:15.466220+0000","last_peered":"2024-09-11T05:14:15.466220+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:15.466220+0000","last_undegraded":"2024-09-11T05:14:15.466220+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14458880,"num_objects":3530,"num_object_clones":0,"num_object_copies":7060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":3530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1439,"num_write_kb":5756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":102400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3453},{"shards":"2","objects":77}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3807","reported_seq":3933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582184+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:18.582184+0000","last_peered":"2024-09-11T05:14:18.582184+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:18.582184+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":3807,"log_dups_size":0,"ondisk_log_size":3807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15585280,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":3360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":182},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3650","reported_seq":3778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.500236+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:18.500236+0000","last_peered":"2024-09-11T05:14:18.500236+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:18.500236+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":3650,"log_dups_size":0,"ondisk_log_size":3650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14917632,"num_objects":3642,"num_object_clones":0,"num_object_copies":7284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":516,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":147},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3713","reported_seq":3843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582519+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:18.582519+0000","last_peered":"2024-09-11T05:14:18.582519+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:18.582519+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":3713,"log_dups_size":0,"ondisk_log_size":3713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15192064,"num_objects":3709,"num_object_clones":0,"num_object_copies":7418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":471,"num_write_kb":1884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":165},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3698","reported_seq":3836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.414176+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:18.414176+0000","last_peered":"2024-09-11T05:14:18.414176+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:18.414176+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":3698,"log_dups_size":0,"ondisk_log_size":3698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15098473,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":705,"num_write_kb":2820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":172},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":240689769,"num_objects":58763,"num_object_clones":0,"num_object_copies":117526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":58763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21908,"num_write_kb":87632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":106496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":478015488,"data_stored":477928513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58844,"ondisk_log_size":58844,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358988,"kb_used_data":228732,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94012852,"statfs":{"total":96636764160,"available":96269160448,"internally_reserved":0,"allocated":234221568,"data_stored":233977109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51104,"kb_used_data":1356,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94320736,"statfs":{"total":96636764160,"available":96584433664,"internally_reserved":0,"allocated":1388544,"data_stored":1063470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":50917035},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215126,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383008,"kb_used_data":234768,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93988832,"statfs":{"total":96636764160,"available":96244563968,"internally_reserved":0,"allocated":240402432,"data_stored":240153896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":240799744,"data_stored":240796265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2998272,"data_stored":2918255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234217472,"data_stored":234213993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:20.081 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 47 20 60443 60423 5.01996 2.52344 0.0344684 0.0155535 2024-09-11T05:14:21.082 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 48 20 61118 61098 4.9703 2.63672 0.0135374 0.015709 2024-09-11T05:14:22.082 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 49 20 61804 61784 4.92356 2.67969 0.0632605 0.0158544 2024-09-11T05:14:22.800 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:23.082 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 50 20 62460 62440 4.87635 2.5625 0.0422338 0.0160128 2024-09-11T05:14:23.709 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:23.709 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:23.769 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-11T05:14:23.374375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":245580412,"num_objects":59959,"num_object_clones":0,"num_object_copies":119918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":59959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23104,"num_write_kb":92410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":557075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60041,"ondisk_log_size":60041,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":823128,"kb_used_data":481000,"kb_used_omap":12,"kb_used_meta":342067,"kb_avail":282292392,"statfs":{"total":289910292480,"available":289067409408,"internally_reserved":0,"allocated":492544000,"data_stored":491725931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":350276609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,8,5,5,4,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26100106,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":-36658,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3566,"num_write_kb":-14229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":544768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002190"},"pg_stats":[{"pgid":"2.f","version":"43'3842","reported_seq":3915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582082+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:18.582082+0000","last_peered":"2024-09-11T05:14:18.582082+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:18.582082+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3842,"log_dups_size":0,"ondisk_log_size":3842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15736832,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3842,"num_write_kb":15368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3787","reported_seq":3884,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:20.409339+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:20.409339+0000","last_peered":"2024-09-11T05:14:20.409339+0000","last_clean":"2024-09-11T05:14:20.409339+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:20.409339+0000","last_undegraded":"2024-09-11T05:14:20.409339+0000","last_fullsized":"2024-09-11T05:14:20.409339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15486976,"num_objects":3781,"num_object_clones":0,"num_object_copies":7562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2185,"num_write_kb":8740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3740","reported_seq":3860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.460632+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:20.460632+0000","last_peered":"2024-09-11T05:14:20.460632+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:20.460632+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3740,"log_dups_size":0,"ondisk_log_size":3740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15302656,"num_objects":3736,"num_object_clones":0,"num_object_copies":7472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2143,"num_write_kb":8572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3884","reported_seq":3950,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:20.485470+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:20.485470+0000","last_peered":"2024-09-11T05:14:20.485470+0000","last_clean":"2024-09-11T05:14:20.485470+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:20.485470+0000","last_undegraded":"2024-09-11T05:14:20.485470+0000","last_fullsized":"2024-09-11T05:14:20.485470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3884,"log_dups_size":0,"ondisk_log_size":3884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15908864,"num_objects":3884,"num_object_clones":0,"num_object_copies":7768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3884,"num_write_kb":15536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3783","reported_seq":3907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:18.552413+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:18.552413+0000","last_peered":"2024-09-11T05:14:18.552413+0000","last_clean":"2024-09-11T05:14:18.552413+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:18.552413+0000","last_undegraded":"2024-09-11T05:14:18.552413+0000","last_fullsized":"2024-09-11T05:14:18.552413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15466496,"num_objects":3776,"num_object_clones":0,"num_object_copies":7552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":2468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3782","reported_seq":3924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.560308+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:18.560308+0000","last_peered":"2024-09-11T05:14:18.560308+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:18.560308+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":3782,"log_dups_size":0,"ondisk_log_size":3782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":326,"num_write_kb":1304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":177},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3756","reported_seq":3883,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.524540+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:18.524540+0000","last_peered":"2024-09-11T05:14:18.524540+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:18.524540+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":3756,"log_dups_size":0,"ondisk_log_size":3756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360000,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":3684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3677","reported_seq":3796,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:18.546577+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:18.546577+0000","last_peered":"2024-09-11T05:14:18.546577+0000","last_clean":"2024-09-11T05:14:18.546577+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:18.546577+0000","last_undegraded":"2024-09-11T05:14:18.546577+0000","last_fullsized":"2024-09-11T05:14:18.546577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":3677,"log_dups_size":0,"ondisk_log_size":3677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15052800,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":980,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3779","reported_seq":3905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.396981+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:20.396981+0000","last_peered":"2024-09-11T05:14:20.396981+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:20.396981+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1847,"num_write_kb":7388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":247}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3623","reported_seq":3780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.572690+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:18.572690+0000","last_peered":"2024-09-11T05:14:18.572690+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:18.572690+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":3623,"log_dups_size":0,"ondisk_log_size":3623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14807040,"num_objects":3615,"num_object_clones":0,"num_object_copies":7230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":173},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3760","reported_seq":3872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.407820+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:20.407820+0000","last_peered":"2024-09-11T05:14:20.407820+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:20.407820+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3760,"log_dups_size":0,"ondisk_log_size":3760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1789,"num_write_kb":7156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3757","reported_seq":4140,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:20.496518+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:20.496518+0000","last_peered":"2024-09-11T05:14:20.496518+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:20.496518+0000","last_undegraded":"2024-09-11T05:14:20.496518+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3757,"log_dups_size":0,"ondisk_log_size":3757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364096,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1660,"num_write_kb":6640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":552960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3343},{"shards":"2","objects":408}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3807","reported_seq":3933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582184+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:18.582184+0000","last_peered":"2024-09-11T05:14:18.582184+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:18.582184+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":3807,"log_dups_size":0,"ondisk_log_size":3807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15585280,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":3360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":182},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3650","reported_seq":3778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.500236+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:18.500236+0000","last_peered":"2024-09-11T05:14:18.500236+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:18.500236+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":3650,"log_dups_size":0,"ondisk_log_size":3650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14917632,"num_objects":3642,"num_object_clones":0,"num_object_copies":7284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":516,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":147},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3713","reported_seq":3843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582519+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:18.582519+0000","last_peered":"2024-09-11T05:14:18.582519+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:18.582519+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":3713,"log_dups_size":0,"ondisk_log_size":3713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15192064,"num_objects":3709,"num_object_clones":0,"num_object_copies":7418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":471,"num_write_kb":1884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":165},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3698","reported_seq":3836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.414176+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:18.414176+0000","last_peered":"2024-09-11T05:14:18.414176+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:18.414176+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":3698,"log_dups_size":0,"ondisk_log_size":3698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15098473,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":705,"num_write_kb":2820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":172},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":245580393,"num_objects":59957,"num_object_clones":0,"num_object_copies":119914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":59957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23102,"num_write_kb":92408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":494792704,"data_stored":494705729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60038,"ondisk_log_size":60038,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370804,"kb_used_data":234980,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94001036,"statfs":{"total":96636764160,"available":96257060864,"internally_reserved":0,"allocated":240619520,"data_stored":240375061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69316,"kb_used_data":11252,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94302524,"statfs":{"total":96636764160,"available":96565784576,"internally_reserved":0,"allocated":11522048,"data_stored":11196974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":59436715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215126,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383008,"kb_used_data":234768,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93988832,"statfs":{"total":96636764160,"available":96244563968,"internally_reserved":0,"allocated":240402432,"data_stored":240153896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":240799744,"data_stored":240796265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13324288,"data_stored":13244271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240668672,"data_stored":240665193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:23.770 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:24.083 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 51 20 63084 63064 4.82841 2.4375 0.0369799 0.0161693 2024-09-11T05:14:24.686 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:24.687 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:24.729 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-11T05:14:23.374375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":245580412,"num_objects":59959,"num_object_clones":0,"num_object_copies":119918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":59959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23104,"num_write_kb":92410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":557075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60041,"ondisk_log_size":60041,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":823128,"kb_used_data":481000,"kb_used_omap":12,"kb_used_meta":342067,"kb_avail":282292392,"statfs":{"total":289910292480,"available":289067409408,"internally_reserved":0,"allocated":492544000,"data_stored":491725931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":350276609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,8,5,5,4,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26100106,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":-36658,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3566,"num_write_kb":-14229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":544768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002190"},"pg_stats":[{"pgid":"2.f","version":"43'3842","reported_seq":3915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582082+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:18.582082+0000","last_peered":"2024-09-11T05:14:18.582082+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:18.582082+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3842,"log_dups_size":0,"ondisk_log_size":3842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15736832,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3842,"num_write_kb":15368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3787","reported_seq":3884,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:20.409339+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:20.409339+0000","last_peered":"2024-09-11T05:14:20.409339+0000","last_clean":"2024-09-11T05:14:20.409339+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:20.409339+0000","last_undegraded":"2024-09-11T05:14:20.409339+0000","last_fullsized":"2024-09-11T05:14:20.409339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15486976,"num_objects":3781,"num_object_clones":0,"num_object_copies":7562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2185,"num_write_kb":8740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3740","reported_seq":3860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.460632+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:20.460632+0000","last_peered":"2024-09-11T05:14:20.460632+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:20.460632+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3740,"log_dups_size":0,"ondisk_log_size":3740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15302656,"num_objects":3736,"num_object_clones":0,"num_object_copies":7472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2143,"num_write_kb":8572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3884","reported_seq":3950,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:20.485470+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:20.485470+0000","last_peered":"2024-09-11T05:14:20.485470+0000","last_clean":"2024-09-11T05:14:20.485470+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:20.485470+0000","last_undegraded":"2024-09-11T05:14:20.485470+0000","last_fullsized":"2024-09-11T05:14:20.485470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3884,"log_dups_size":0,"ondisk_log_size":3884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15908864,"num_objects":3884,"num_object_clones":0,"num_object_copies":7768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3884,"num_write_kb":15536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'3783","reported_seq":3907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:18.552413+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:18.552413+0000","last_peered":"2024-09-11T05:14:18.552413+0000","last_clean":"2024-09-11T05:14:18.552413+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:18.552413+0000","last_undegraded":"2024-09-11T05:14:18.552413+0000","last_fullsized":"2024-09-11T05:14:18.552413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":3783,"log_dups_size":0,"ondisk_log_size":3783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15466496,"num_objects":3776,"num_object_clones":0,"num_object_copies":7552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":2468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3782","reported_seq":3924,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.560308+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:18.560308+0000","last_peered":"2024-09-11T05:14:18.560308+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:18.560308+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":3782,"log_dups_size":0,"ondisk_log_size":3782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":326,"num_write_kb":1304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":177},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3756","reported_seq":3883,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.524540+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:18.524540+0000","last_peered":"2024-09-11T05:14:18.524540+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:18.524540+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":3756,"log_dups_size":0,"ondisk_log_size":3756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360000,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":3684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3677","reported_seq":3796,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:18.546577+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:18.546577+0000","last_peered":"2024-09-11T05:14:18.546577+0000","last_clean":"2024-09-11T05:14:18.546577+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:18.546577+0000","last_undegraded":"2024-09-11T05:14:18.546577+0000","last_fullsized":"2024-09-11T05:14:18.546577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":3677,"log_dups_size":0,"ondisk_log_size":3677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15052800,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":980,"num_write_kb":3920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3779","reported_seq":3905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.396981+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:20.396981+0000","last_peered":"2024-09-11T05:14:20.396981+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:20.396981+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1847,"num_write_kb":7388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":247}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3623","reported_seq":3780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.572690+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:18.572690+0000","last_peered":"2024-09-11T05:14:18.572690+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:18.572690+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":3623,"log_dups_size":0,"ondisk_log_size":3623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14807040,"num_objects":3615,"num_object_clones":0,"num_object_copies":7230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":376,"num_write_kb":1504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":173},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3760","reported_seq":3872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.407820+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:20.407820+0000","last_peered":"2024-09-11T05:14:20.407820+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:20.407820+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3760,"log_dups_size":0,"ondisk_log_size":3760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1789,"num_write_kb":7156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3757","reported_seq":4140,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:20.496518+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:20.496518+0000","last_peered":"2024-09-11T05:14:20.496518+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:20.496518+0000","last_undegraded":"2024-09-11T05:14:20.496518+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3757,"log_dups_size":0,"ondisk_log_size":3757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364096,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1660,"num_write_kb":6640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":552960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3343},{"shards":"2","objects":408}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'3807","reported_seq":3933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582184+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:18.582184+0000","last_peered":"2024-09-11T05:14:18.582184+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:18.582184+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":3807,"log_dups_size":0,"ondisk_log_size":3807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15585280,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":3360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":182},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3650","reported_seq":3778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.500236+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:18.500236+0000","last_peered":"2024-09-11T05:14:18.500236+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:18.500236+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":3650,"log_dups_size":0,"ondisk_log_size":3650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14917632,"num_objects":3642,"num_object_clones":0,"num_object_copies":7284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":516,"num_write_kb":2064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":147},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3713","reported_seq":3843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.582519+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:18.582519+0000","last_peered":"2024-09-11T05:14:18.582519+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:18.582519+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":3713,"log_dups_size":0,"ondisk_log_size":3713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15192064,"num_objects":3709,"num_object_clones":0,"num_object_copies":7418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":471,"num_write_kb":1884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":165},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3698","reported_seq":3836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:18.414176+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:18.414176+0000","last_peered":"2024-09-11T05:14:18.414176+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:18.414176+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":3698,"log_dups_size":0,"ondisk_log_size":3698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15098473,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":705,"num_write_kb":2820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":172},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":245580393,"num_objects":59957,"num_object_clones":0,"num_object_copies":119914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":59957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23102,"num_write_kb":92408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":494792704,"data_stored":494705729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60038,"ondisk_log_size":60038,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370804,"kb_used_data":234980,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94001036,"statfs":{"total":96636764160,"available":96257060864,"internally_reserved":0,"allocated":240619520,"data_stored":240375061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69316,"kb_used_data":11252,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94302524,"statfs":{"total":96636764160,"available":96565784576,"internally_reserved":0,"allocated":11522048,"data_stored":11196974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":59436715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215126,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383008,"kb_used_data":234768,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93988832,"statfs":{"total":96636764160,"available":96244563968,"internally_reserved":0,"allocated":240402432,"data_stored":240153896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":240799744,"data_stored":240796265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13324288,"data_stored":13244271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240668672,"data_stored":240665193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'3842", 'reported_seq': 3915, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.582082+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:14:18.582082+0000', 'last_peered': '2024-09-11T05:14:18.582082+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:14:18.582082+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 3842, 'log_dups_size': 0, 'ondisk_log_size': 3842, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15736832, 'num_objects': 3842, 'num_object_clones': 0, 'num_object_copies': 7684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3842, 'num_write_kb': 15368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 167}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'3740", 'reported_seq': 3860, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:20.460632+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:14:20.460632+0000', 'last_peered': '2024-09-11T05:14:20.460632+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:14:20.460632+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 3740, 'log_dups_size': 0, 'ondisk_log_size': 3740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15302656, 'num_objects': 3736, 'num_object_clones': 0, 'num_object_copies': 7472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2143, 'num_write_kb': 8572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 242}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'3782", 'reported_seq': 3924, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.560308+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:14:18.560308+0000', 'last_peered': '2024-09-11T05:14:18.560308+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:14:18.560308+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 3782, 'log_dups_size': 0, 'ondisk_log_size': 3782, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15470592, 'num_objects': 3777, 'num_object_clones': 0, 'num_object_copies': 7554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 326, 'num_write_kb': 1304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 177}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'3756", 'reported_seq': 3883, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.524540+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:14:18.524540+0000', 'last_peered': '2024-09-11T05:14:18.524540+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:14:18.524540+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 3756, 'log_dups_size': 0, 'ondisk_log_size': 3756, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15360000, 'num_objects': 3750, 'num_object_clones': 0, 'num_object_copies': 7500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 921, 'num_write_kb': 3684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 174}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:14:24.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'3779", 'reported_seq': 3905, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:20.396981+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:14:20.396981+0000', 'last_peered': '2024-09-11T05:14:20.396981+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:14:20.396981+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 3779, 'log_dups_size': 0, 'ondisk_log_size': 3779, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15450112, 'num_objects': 3772, 'num_object_clones': 0, 'num_object_copies': 7544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1847, 'num_write_kb': 7388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 247}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:24.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:14:24.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'3623", 'reported_seq': 3780, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.572690+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:14:18.572690+0000', 'last_peered': '2024-09-11T05:14:18.572690+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:14:18.572690+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 3623, 'log_dups_size': 0, 'ondisk_log_size': 3623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14807040, 'num_objects': 3615, 'num_object_clones': 0, 'num_object_copies': 7230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 376, 'num_write_kb': 1504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 173}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:14:24.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'3760", 'reported_seq': 3872, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:20.407820+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:14:20.407820+0000', 'last_peered': '2024-09-11T05:14:20.407820+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:14:20.407820+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 3760, 'log_dups_size': 0, 'ondisk_log_size': 3760, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15380480, 'num_objects': 3755, 'num_object_clones': 0, 'num_object_copies': 7510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1789, 'num_write_kb': 7156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 233}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:24.732 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'3757", 'reported_seq': 4140, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:14:20.496518+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:14:20.496518+0000', 'last_peered': '2024-09-11T05:14:20.496518+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:14:20.496518+0000', 'last_undegraded': '2024-09-11T05:14:20.496518+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 3757, 'log_dups_size': 0, 'ondisk_log_size': 3757, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15364096, 'num_objects': 3751, 'num_object_clones': 0, 'num_object_copies': 7502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3343, 'num_objects_unfound': 0, 'num_objects_dirty': 3751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1660, 'num_write_kb': 6640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 135, 'num_bytes_recovered': 552960, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3343}, {'shards': '2', 'objects': 408}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'3807", 'reported_seq': 3933, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.582184+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:14:18.582184+0000', 'last_peered': '2024-09-11T05:14:18.582184+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:14:18.582184+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 3807, 'log_dups_size': 0, 'ondisk_log_size': 3807, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15585280, 'num_objects': 3805, 'num_object_clones': 0, 'num_object_copies': 7610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 840, 'num_write_kb': 3360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 182}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'3650", 'reported_seq': 3778, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.500236+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:14:18.500236+0000', 'last_peered': '2024-09-11T05:14:18.500236+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:14:18.500236+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 3650, 'log_dups_size': 0, 'ondisk_log_size': 3650, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14917632, 'num_objects': 3642, 'num_object_clones': 0, 'num_object_copies': 7284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3642, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 516, 'num_write_kb': 2064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 147}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'3713", 'reported_seq': 3843, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.582519+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:14:18.582519+0000', 'last_peered': '2024-09-11T05:14:18.582519+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:14:18.582519+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 3713, 'log_dups_size': 0, 'ondisk_log_size': 3713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15192064, 'num_objects': 3709, 'num_object_clones': 0, 'num_object_copies': 7418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 471, 'num_write_kb': 1884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 165}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:14:24.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'3698", 'reported_seq': 3836, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:18.414176+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:14:18.414176+0000', 'last_peered': '2024-09-11T05:14:18.414176+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:14:18.414176+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 3698, 'log_dups_size': 0, 'ondisk_log_size': 3698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15098473, 'num_objects': 3687, 'num_object_clones': 0, 'num_object_copies': 7374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3687, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 705, 'num_write_kb': 2820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 172}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:24.734 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:14:25.085 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 52 20 63702 63682 4.7819 2.41406 0.00582137 0.0163244 2024-09-11T05:14:25.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:25.868 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/123922 objects degraded (31.496%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":143,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":61961,"data_bytes":253780604,"bytes_used":844603392,"bytes_avail":289065689088,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":123922,"degraded_ratio":0.31496425170671877,"misplaced_objects":3343,"misplaced_total":123922,"misplaced_ratio":0.026976646600280822,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":47499,"recovering_keys_per_sec":0,"num_objects_recovered":135,"num_bytes_recovered":552960,"num_keys_recovered":0,"write_bytes_sec":2069562,"read_op_per_sec":0,"write_op_per_sec":505},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:14:25.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:14:25.954 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:26.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 53 20 64315 64295 4.73669 2.39453 0.0194436 0.0164848 2024-09-11T05:14:26.676 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:26.732 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-11T05:14:25.374789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":253780604,"num_objects":61961,"num_object_clones":0,"num_object_copies":123922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":61961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25106,"num_write_kb":100418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":557075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62043,"ondisk_log_size":62043,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":824808,"kb_used_data":491000,"kb_used_omap":12,"kb_used_meta":333747,"kb_avail":282290712,"statfs":{"total":289910292480,"available":289065689088,"internally_reserved":0,"allocated":502784000,"data_stored":501965931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":341756929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,4,7,2,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24092672,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":-10205,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":552960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.641430"},"pg_stats":[{"pgid":"2.f","version":"43'4034","reported_seq":4107,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.563579+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:23.563579+0000","last_peered":"2024-09-11T05:14:23.563579+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:23.563579+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4034,"num_write_kb":16136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":359},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3787","reported_seq":3884,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:20.409339+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:20.409339+0000","last_peered":"2024-09-11T05:14:20.409339+0000","last_clean":"2024-09-11T05:14:20.409339+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:20.409339+0000","last_undegraded":"2024-09-11T05:14:20.409339+0000","last_fullsized":"2024-09-11T05:14:20.409339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3787,"log_dups_size":0,"ondisk_log_size":3787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15486976,"num_objects":3781,"num_object_clones":0,"num_object_copies":7562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2185,"num_write_kb":8740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3740","reported_seq":3860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.460632+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:20.460632+0000","last_peered":"2024-09-11T05:14:20.460632+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:20.460632+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3740,"log_dups_size":0,"ondisk_log_size":3740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15302656,"num_objects":3736,"num_object_clones":0,"num_object_copies":7472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2143,"num_write_kb":8572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3884","reported_seq":3950,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:20.485470+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:20.485470+0000","last_peered":"2024-09-11T05:14:20.485470+0000","last_clean":"2024-09-11T05:14:20.485470+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:20.485470+0000","last_undegraded":"2024-09-11T05:14:20.485470+0000","last_fullsized":"2024-09-11T05:14:20.485470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":3884,"log_dups_size":0,"ondisk_log_size":3884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15908864,"num_objects":3884,"num_object_clones":0,"num_object_copies":7768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3884,"num_write_kb":15536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4000","reported_seq":4124,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:23.510962+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:23.510962+0000","last_peered":"2024-09-11T05:14:23.510962+0000","last_clean":"2024-09-11T05:14:23.510962+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:23.510962+0000","last_undegraded":"2024-09-11T05:14:23.510962+0000","last_fullsized":"2024-09-11T05:14:23.510962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4000,"log_dups_size":0,"ondisk_log_size":4000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16355328,"num_objects":3993,"num_object_clones":0,"num_object_copies":7986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":834,"num_write_kb":3336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'3965","reported_seq":4109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.585196+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:23.585196+0000","last_peered":"2024-09-11T05:14:23.585196+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:23.585196+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":3965,"log_dups_size":0,"ondisk_log_size":3965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":509,"num_write_kb":2036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":360},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'3950","reported_seq":4076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.552105+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:23.552105+0000","last_peered":"2024-09-11T05:14:23.552105+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:23.552105+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":3950,"log_dups_size":0,"ondisk_log_size":3950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16154624,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":4460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":368},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3883","reported_seq":4002,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:23.571966+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:23.571966+0000","last_peered":"2024-09-11T05:14:23.571966+0000","last_clean":"2024-09-11T05:14:23.571966+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:23.571966+0000","last_undegraded":"2024-09-11T05:14:23.571966+0000","last_fullsized":"2024-09-11T05:14:23.571966+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":3883,"log_dups_size":0,"ondisk_log_size":3883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15896576,"num_objects":3881,"num_object_clones":0,"num_object_copies":7762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1186,"num_write_kb":4744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3779","reported_seq":3905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.396981+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:20.396981+0000","last_peered":"2024-09-11T05:14:20.396981+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:20.396981+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3779,"log_dups_size":0,"ondisk_log_size":3779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450112,"num_objects":3772,"num_object_clones":0,"num_object_copies":7544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1847,"num_write_kb":7388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":247}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'3809","reported_seq":3969,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.566577+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:23.566577+0000","last_peered":"2024-09-11T05:14:23.566577+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:23.566577+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":3809,"log_dups_size":0,"ondisk_log_size":3809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15568896,"num_objects":3801,"num_object_clones":0,"num_object_copies":7602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":562,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":359},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3760","reported_seq":3872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:20.407820+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:20.407820+0000","last_peered":"2024-09-11T05:14:20.407820+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:20.407820+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3760,"log_dups_size":0,"ondisk_log_size":3760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1789,"num_write_kb":7156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":233}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3757","reported_seq":4140,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:20.496518+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:20.496518+0000","last_peered":"2024-09-11T05:14:20.496518+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:20.496518+0000","last_undegraded":"2024-09-11T05:14:20.496518+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3757,"log_dups_size":0,"ondisk_log_size":3757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364096,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1660,"num_write_kb":6640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":552960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3343},{"shards":"2","objects":408}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4024","reported_seq":4150,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.551995+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:23.551995+0000","last_peered":"2024-09-11T05:14:23.551995+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:23.551995+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4024,"log_dups_size":0,"ondisk_log_size":4024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16474112,"num_objects":4022,"num_object_clones":0,"num_object_copies":8044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1057,"num_write_kb":4228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":399},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'3848","reported_seq":3974,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.594678+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:23.594678+0000","last_peered":"2024-09-11T05:14:23.594678+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:23.594678+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":3848,"log_dups_size":0,"ondisk_log_size":3848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":714,"num_write_kb":2856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":345},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'3911","reported_seq":4041,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.564556+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:23.564556+0000","last_peered":"2024-09-11T05:14:23.564556+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:23.564556+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":3911,"log_dups_size":0,"ondisk_log_size":3911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16003072,"num_objects":3907,"num_object_clones":0,"num_object_copies":7814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":669,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":363},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'3909","reported_seq":4047,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:23.605962+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:23.605962+0000","last_peered":"2024-09-11T05:14:23.605962+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:23.605962+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962729,"num_objects":3898,"num_object_clones":0,"num_object_copies":7796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":916,"num_write_kb":3664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":383},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":253780585,"num_objects":61959,"num_object_clones":0,"num_object_copies":123918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3343,"num_objects_unfound":0,"num_objects_dirty":61959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25104,"num_write_kb":100416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":504926208,"data_stored":504839233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62040,"ondisk_log_size":62040,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370804,"kb_used_data":234980,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94001036,"statfs":{"total":96636764160,"available":96257060864,"internally_reserved":0,"allocated":240619520,"data_stored":240375061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69316,"kb_used_data":11252,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94302524,"statfs":{"total":96636764160,"available":96565784576,"internally_reserved":0,"allocated":11522048,"data_stored":11196974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":59436715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215127,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384688,"kb_used_data":244768,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93987152,"statfs":{"total":96636764160,"available":96242843648,"internally_reserved":0,"allocated":250642432,"data_stored":250393896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143257259},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,3,0,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250933248,"data_stored":250929769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13324288,"data_stored":13244271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240668672,"data_stored":240665193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:27.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 54 20 64947 64927 4.6947 2.46875 0.0717224 0.0166295 2024-09-11T05:14:28.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 55 20 65619 65599 4.65708 2.625 0.0785946 0.0167635 2024-09-11T05:14:29.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 56 20 66261 66241 4.61857 2.50781 0.0227884 0.0169053 2024-09-11T05:14:29.734 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:30.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 57 20 66892 66872 4.5808 2.46484 0.0257663 0.0170437 2024-09-11T05:14:30.620 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:30.623 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:30.668 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-11T05:14:29.375738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266941052,"num_objects":65174,"num_object_clones":0,"num_object_copies":130348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3224,"num_objects_unfound":0,"num_objects_dirty":65174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28319,"num_write_kb":113270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":1044499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65256,"ondisk_log_size":65256,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":851304,"kb_used_data":517496,"kb_used_omap":12,"kb_used_meta":333747,"kb_avail":282264216,"statfs":{"total":289910292480,"available":289038557184,"internally_reserved":0,"allocated":529915904,"data_stored":529097835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":341756929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,7,6,13,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28485,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8103,"num_write_kb":32412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002781"},"pg_stats":[{"pgid":"2.f","version":"43'4234","reported_seq":4307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.622556+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:28.622556+0000","last_peered":"2024-09-11T05:14:28.622556+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:28.622556+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4234,"log_dups_size":0,"ondisk_log_size":4234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4234,"num_write_kb":16936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":559},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3987","reported_seq":4084,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:25.415083+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:25.415083+0000","last_peered":"2024-09-11T05:14:25.415083+0000","last_clean":"2024-09-11T05:14:25.415083+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:25.415083+0000","last_undegraded":"2024-09-11T05:14:25.415083+0000","last_fullsized":"2024-09-11T05:14:25.415083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3987,"log_dups_size":0,"ondisk_log_size":3987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3959","reported_seq":4079,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:25.450035+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:25.450035+0000","last_peered":"2024-09-11T05:14:25.450035+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:25.450035+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3959,"log_dups_size":0,"ondisk_log_size":3959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":461}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4110","reported_seq":4176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:25.453468+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:25.453468+0000","last_peered":"2024-09-11T05:14:25.453468+0000","last_clean":"2024-09-11T05:14:25.453468+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:25.453468+0000","last_undegraded":"2024-09-11T05:14:25.453468+0000","last_fullsized":"2024-09-11T05:14:25.453468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4110,"log_dups_size":0,"ondisk_log_size":4110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4212","reported_seq":4336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:28.644901+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:28.644901+0000","last_peered":"2024-09-11T05:14:28.644901+0000","last_clean":"2024-09-11T05:14:28.644901+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:28.644901+0000","last_undegraded":"2024-09-11T05:14:28.644901+0000","last_fullsized":"2024-09-11T05:14:28.644901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17223680,"num_objects":4205,"num_object_clones":0,"num_object_copies":8410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1046,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4148","reported_seq":4292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.617028+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:28.617028+0000","last_peered":"2024-09-11T05:14:28.617028+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:28.617028+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4148,"log_dups_size":0,"ondisk_log_size":4148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16969728,"num_objects":4143,"num_object_clones":0,"num_object_copies":8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":692,"num_write_kb":2768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":543},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4151","reported_seq":4278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.634197+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:28.634197+0000","last_peered":"2024-09-11T05:14:28.634197+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:28.634197+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4151,"log_dups_size":0,"ondisk_log_size":4151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1316,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4070","reported_seq":4188,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:28.627585+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:28.627585+0000","last_peered":"2024-09-11T05:14:28.627585+0000","last_clean":"2024-09-11T05:14:28.627585+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:28.627585+0000","last_undegraded":"2024-09-11T05:14:28.627585+0000","last_fullsized":"2024-09-11T05:14:28.627585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16662528,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1373,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3998","reported_seq":4124,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:25.501102+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:25.501102+0000","last_peered":"2024-09-11T05:14:25.501102+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:25.501102+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16347136,"num_objects":3991,"num_object_clones":0,"num_object_copies":7982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2066,"num_write_kb":8264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4012","reported_seq":4172,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.590769+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:28.590769+0000","last_peered":"2024-09-11T05:14:28.590769+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:28.590769+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4012,"log_dups_size":0,"ondisk_log_size":4012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":562},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3950","reported_seq":4062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:25.363112+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:25.363112+0000","last_peered":"2024-09-11T05:14:25.363112+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:25.363112+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3950,"log_dups_size":0,"ondisk_log_size":3950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16158720,"num_objects":3945,"num_object_clones":0,"num_object_copies":7890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1979,"num_write_kb":7916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3968","reported_seq":4589,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:25.495215+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:25.495215+0000","last_peered":"2024-09-11T05:14:25.495215+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:25.495215+0000","last_undegraded":"2024-09-11T05:14:25.495215+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3968,"log_dups_size":0,"ondisk_log_size":3968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3224,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1871,"num_write_kb":7484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3224},{"shards":"2","objects":738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4231","reported_seq":4356,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.659447+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:28.659447+0000","last_peered":"2024-09-11T05:14:28.659447+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:28.659447+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17321984,"num_objects":4229,"num_object_clones":0,"num_object_copies":8458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1264,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":606},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4051","reported_seq":4179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.619414+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:28.619414+0000","last_peered":"2024-09-11T05:14:28.619414+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:28.619414+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":917,"num_write_kb":3668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":548},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4072","reported_seq":4202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.640222+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:28.640222+0000","last_peered":"2024-09-11T05:14:28.640222+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:28.640222+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4072,"log_dups_size":0,"ondisk_log_size":4072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16662528,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":830,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":524},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4100","reported_seq":4238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.636189+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:28.636189+0000","last_peered":"2024-09-11T05:14:28.636189+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:28.636189+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16745065,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1107,"num_write_kb":4428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":574},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266941033,"num_objects":65172,"num_object_clones":0,"num_object_copies":130344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3224,"num_objects_unfound":0,"num_objects_dirty":65172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28317,"num_write_kb":113268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":1044480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":531808256,"data_stored":531721281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65253,"ondisk_log_size":65253,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381480,"kb_used_data":241496,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93990360,"statfs":{"total":96636764160,"available":96246128640,"internally_reserved":0,"allocated":247291904,"data_stored":247047445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71420,"kb_used_data":21676,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94300420,"statfs":{"total":96636764160,"available":96563630080,"internally_reserved":0,"allocated":22196224,"data_stored":21871150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":50917035},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215128,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398404,"kb_used_data":254324,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973436,"statfs":{"total":96636764160,"available":96228798464,"internally_reserved":0,"allocated":260427776,"data_stored":260179240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,6,3,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260644864,"data_stored":260641385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23678976,"data_stored":23598959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247484416,"data_stored":247480937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:30.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:31.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 58 20 67483 67463 4.54163 2.30859 0.0127133 0.0171906 2024-09-11T05:14:31.343 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:31.344 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:31.405 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-11T05:14:29.375738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266941052,"num_objects":65174,"num_object_clones":0,"num_object_copies":130348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3224,"num_objects_unfound":0,"num_objects_dirty":65174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28319,"num_write_kb":113270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":1044499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65256,"ondisk_log_size":65256,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":851304,"kb_used_data":517496,"kb_used_omap":12,"kb_used_meta":333747,"kb_avail":282264216,"statfs":{"total":289910292480,"available":289038557184,"internally_reserved":0,"allocated":529915904,"data_stored":529097835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":341756929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,7,6,13,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28485,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8103,"num_write_kb":32412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002781"},"pg_stats":[{"pgid":"2.f","version":"43'4234","reported_seq":4307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.622556+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:28.622556+0000","last_peered":"2024-09-11T05:14:28.622556+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:28.622556+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4234,"log_dups_size":0,"ondisk_log_size":4234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4234,"num_write_kb":16936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":559},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'3987","reported_seq":4084,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:25.415083+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:25.415083+0000","last_peered":"2024-09-11T05:14:25.415083+0000","last_clean":"2024-09-11T05:14:25.415083+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:25.415083+0000","last_undegraded":"2024-09-11T05:14:25.415083+0000","last_fullsized":"2024-09-11T05:14:25.415083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":3987,"log_dups_size":0,"ondisk_log_size":3987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2385,"num_write_kb":9540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'3959","reported_seq":4079,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:25.450035+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:25.450035+0000","last_peered":"2024-09-11T05:14:25.450035+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:25.450035+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":3959,"log_dups_size":0,"ondisk_log_size":3959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":461}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4110","reported_seq":4176,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:25.453468+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:25.453468+0000","last_peered":"2024-09-11T05:14:25.453468+0000","last_clean":"2024-09-11T05:14:25.453468+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:25.453468+0000","last_undegraded":"2024-09-11T05:14:25.453468+0000","last_fullsized":"2024-09-11T05:14:25.453468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4110,"log_dups_size":0,"ondisk_log_size":4110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4212","reported_seq":4336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:28.644901+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:28.644901+0000","last_peered":"2024-09-11T05:14:28.644901+0000","last_clean":"2024-09-11T05:14:28.644901+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:28.644901+0000","last_undegraded":"2024-09-11T05:14:28.644901+0000","last_fullsized":"2024-09-11T05:14:28.644901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17223680,"num_objects":4205,"num_object_clones":0,"num_object_copies":8410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1046,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4148","reported_seq":4292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.617028+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:28.617028+0000","last_peered":"2024-09-11T05:14:28.617028+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:28.617028+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4148,"log_dups_size":0,"ondisk_log_size":4148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16969728,"num_objects":4143,"num_object_clones":0,"num_object_copies":8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":692,"num_write_kb":2768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":543},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4151","reported_seq":4278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.634197+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:28.634197+0000","last_peered":"2024-09-11T05:14:28.634197+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:28.634197+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4151,"log_dups_size":0,"ondisk_log_size":4151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1316,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4070","reported_seq":4188,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:28.627585+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:28.627585+0000","last_peered":"2024-09-11T05:14:28.627585+0000","last_clean":"2024-09-11T05:14:28.627585+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:28.627585+0000","last_undegraded":"2024-09-11T05:14:28.627585+0000","last_fullsized":"2024-09-11T05:14:28.627585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16662528,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1373,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'3998","reported_seq":4124,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:25.501102+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:25.501102+0000","last_peered":"2024-09-11T05:14:25.501102+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:25.501102+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":3998,"log_dups_size":0,"ondisk_log_size":3998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16347136,"num_objects":3991,"num_object_clones":0,"num_object_copies":7982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2066,"num_write_kb":8264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4012","reported_seq":4172,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.590769+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:28.590769+0000","last_peered":"2024-09-11T05:14:28.590769+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:28.590769+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4012,"log_dups_size":0,"ondisk_log_size":4012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":562},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'3950","reported_seq":4062,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:25.363112+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:25.363112+0000","last_peered":"2024-09-11T05:14:25.363112+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:25.363112+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":3950,"log_dups_size":0,"ondisk_log_size":3950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16158720,"num_objects":3945,"num_object_clones":0,"num_object_copies":7890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1979,"num_write_kb":7916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'3968","reported_seq":4589,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:25.495215+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:25.495215+0000","last_peered":"2024-09-11T05:14:25.495215+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:25.495215+0000","last_undegraded":"2024-09-11T05:14:25.495215+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":3968,"log_dups_size":0,"ondisk_log_size":3968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3224,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1871,"num_write_kb":7484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3224},{"shards":"2","objects":738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4231","reported_seq":4356,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.659447+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:28.659447+0000","last_peered":"2024-09-11T05:14:28.659447+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:28.659447+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17321984,"num_objects":4229,"num_object_clones":0,"num_object_copies":8458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1264,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":606},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4051","reported_seq":4179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.619414+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:28.619414+0000","last_peered":"2024-09-11T05:14:28.619414+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:28.619414+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":917,"num_write_kb":3668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":548},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4072","reported_seq":4202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.640222+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:28.640222+0000","last_peered":"2024-09-11T05:14:28.640222+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:28.640222+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4072,"log_dups_size":0,"ondisk_log_size":4072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16662528,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":830,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":524},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4100","reported_seq":4238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.636189+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:28.636189+0000","last_peered":"2024-09-11T05:14:28.636189+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:28.636189+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16745065,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1107,"num_write_kb":4428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":574},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266941033,"num_objects":65172,"num_object_clones":0,"num_object_copies":130344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3224,"num_objects_unfound":0,"num_objects_dirty":65172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28317,"num_write_kb":113268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":1044480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":531808256,"data_stored":531721281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65253,"ondisk_log_size":65253,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381480,"kb_used_data":241496,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93990360,"statfs":{"total":96636764160,"available":96246128640,"internally_reserved":0,"allocated":247291904,"data_stored":247047445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71420,"kb_used_data":21676,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94300420,"statfs":{"total":96636764160,"available":96563630080,"internally_reserved":0,"allocated":22196224,"data_stored":21871150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":50917035},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215128,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398404,"kb_used_data":254324,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973436,"statfs":{"total":96636764160,"available":96228798464,"internally_reserved":0,"allocated":260427776,"data_stored":260179240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,6,3,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260644864,"data_stored":260641385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23678976,"data_stored":23598959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247484416,"data_stored":247480937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:31.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4234", 'reported_seq': 4307, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.622556+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:14:28.622556+0000', 'last_peered': '2024-09-11T05:14:28.622556+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:14:28.622556+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 4234, 'log_dups_size': 0, 'ondisk_log_size': 4234, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17342464, 'num_objects': 4234, 'num_object_clones': 0, 'num_object_copies': 8468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4234, 'num_write_kb': 16936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 559}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'3959", 'reported_seq': 4079, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:25.450035+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:14:25.450035+0000', 'last_peered': '2024-09-11T05:14:25.450035+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:14:25.450035+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 3959, 'log_dups_size': 0, 'ondisk_log_size': 3959, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16199680, 'num_objects': 3955, 'num_object_clones': 0, 'num_object_copies': 7910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3955, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2362, 'num_write_kb': 9448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 461}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4148", 'reported_seq': 4292, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.617028+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:14:28.617028+0000', 'last_peered': '2024-09-11T05:14:28.617028+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:14:28.617028+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 4148, 'log_dups_size': 0, 'ondisk_log_size': 4148, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16969728, 'num_objects': 4143, 'num_object_clones': 0, 'num_object_copies': 8286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4143, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 692, 'num_write_kb': 2768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 543}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4151", 'reported_seq': 4278, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.634197+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:14:28.634197+0000', 'last_peered': '2024-09-11T05:14:28.634197+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:14:28.634197+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 4151, 'log_dups_size': 0, 'ondisk_log_size': 4151, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16977920, 'num_objects': 4145, 'num_object_clones': 0, 'num_object_copies': 8290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1316, 'num_write_kb': 5264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 569}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:14:31.407 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'3998", 'reported_seq': 4124, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:25.501102+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:14:25.501102+0000', 'last_peered': '2024-09-11T05:14:25.501102+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:14:25.501102+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 3998, 'log_dups_size': 0, 'ondisk_log_size': 3998, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16347136, 'num_objects': 3991, 'num_object_clones': 0, 'num_object_copies': 7982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2066, 'num_write_kb': 8264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4012", 'reported_seq': 4172, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.590769+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:14:28.590769+0000', 'last_peered': '2024-09-11T05:14:28.590769+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:14:28.590769+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 4012, 'log_dups_size': 0, 'ondisk_log_size': 4012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16400384, 'num_objects': 4004, 'num_object_clones': 0, 'num_object_copies': 8008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 765, 'num_write_kb': 3060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 562}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'3950", 'reported_seq': 4062, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:25.363112+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:14:25.363112+0000', 'last_peered': '2024-09-11T05:14:25.363112+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:14:25.363112+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 3950, 'log_dups_size': 0, 'ondisk_log_size': 3950, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16158720, 'num_objects': 3945, 'num_object_clones': 0, 'num_object_copies': 7890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1979, 'num_write_kb': 7916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 423}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'3968", 'reported_seq': 4589, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:14:25.495215+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:14:25.495215+0000', 'last_peered': '2024-09-11T05:14:25.495215+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:14:25.495215+0000', 'last_undegraded': '2024-09-11T05:14:25.495215+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 3968, 'log_dups_size': 0, 'ondisk_log_size': 3968, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16228352, 'num_objects': 3962, 'num_object_clones': 0, 'num_object_copies': 7924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3224, 'num_objects_unfound': 0, 'num_objects_dirty': 3962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1871, 'num_write_kb': 7484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 254, 'num_bytes_recovered': 1040384, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3224}, {'shards': '2', 'objects': 738}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4231", 'reported_seq': 4356, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.659447+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:14:28.659447+0000', 'last_peered': '2024-09-11T05:14:28.659447+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:14:28.659447+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 4231, 'log_dups_size': 0, 'ondisk_log_size': 4231, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17321984, 'num_objects': 4229, 'num_object_clones': 0, 'num_object_copies': 8458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1264, 'num_write_kb': 5056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 606}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4051", 'reported_seq': 4179, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.619414+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:14:28.619414+0000', 'last_peered': '2024-09-11T05:14:28.619414+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:14:28.619414+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 4051, 'log_dups_size': 0, 'ondisk_log_size': 4051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16560128, 'num_objects': 4043, 'num_object_clones': 0, 'num_object_copies': 8086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4043, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 917, 'num_write_kb': 3668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 548}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.408 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:14:31.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4072", 'reported_seq': 4202, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.640222+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:14:28.640222+0000', 'last_peered': '2024-09-11T05:14:28.640222+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:14:28.640222+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 4072, 'log_dups_size': 0, 'ondisk_log_size': 4072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16662528, 'num_objects': 4068, 'num_object_clones': 0, 'num_object_copies': 8136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 830, 'num_write_kb': 3320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 524}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:14:31.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4100", 'reported_seq': 4238, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:28.636189+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:14:28.636189+0000', 'last_peered': '2024-09-11T05:14:28.636189+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:14:28.636189+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 4100, 'log_dups_size': 0, 'ondisk_log_size': 4100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16745065, 'num_objects': 4089, 'num_object_clones': 0, 'num_object_copies': 8178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1107, 'num_write_kb': 4428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 574}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:31.412 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:14:32.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 59 20 68084 68064 4.50431 2.34766 0.038166 0.0173348 2024-09-11T05:14:32.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:32.554 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/132842 objects degraded (29.382%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":150,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66421,"data_bytes":272048764,"bytes_used":905662464,"bytes_avail":289004630016,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":132842,"degraded_ratio":0.29381520904533204,"misplaced_objects":3092,"misplaced_total":132842,"misplaced_ratio":0.023275771216934402,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":123192,"recovering_keys_per_sec":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"write_bytes_sec":2612646,"read_op_per_sec":0,"write_op_per_sec":637},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:14:32.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:14:32.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:33.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:14:33.092277+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0174766 2024-09-11T05:14:33.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:14:33.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 60 20 68672 68652 4.46753 2.29688 0.028958 0.0174766 2024-09-11T05:14:33.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:33.641 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-11T05:14:33.376649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":272048764,"num_objects":66421,"num_object_clones":0,"num_object_copies":132842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3092,"num_objects_unfound":0,"num_objects_dirty":66421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29566,"num_write_kb":118258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":1585171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66503,"ondisk_log_size":66503,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":884436,"kb_used_data":533988,"kb_used_omap":12,"kb_used_meta":350387,"kb_avail":282231084,"statfs":{"total":289910292480,"available":289004630016,"internally_reserved":0,"allocated":546803712,"data_stored":545985643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":358796289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,7,8,5,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26468352,"num_objects":6462,"num_object_clones":0,"num_object_copies":12924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-251,"num_objects_unfound":0,"num_objects_dirty":6462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6462,"num_write_kb":25848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002741"},"pg_stats":[{"pgid":"2.f","version":"43'4234","reported_seq":4307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.622556+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:28.622556+0000","last_peered":"2024-09-11T05:14:28.622556+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:28.622556+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4234,"log_dups_size":0,"ondisk_log_size":4234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4234,"num_write_kb":16936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":559},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4212","reported_seq":4309,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:30.526958+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:30.526958+0000","last_peered":"2024-09-11T05:14:30.526958+0000","last_clean":"2024-09-11T05:14:30.526958+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:30.526958+0000","last_undegraded":"2024-09-11T05:14:30.526958+0000","last_fullsized":"2024-09-11T05:14:30.526958+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17227776,"num_objects":4206,"num_object_clones":0,"num_object_copies":8412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2610,"num_write_kb":10440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4174","reported_seq":4292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:30.494762+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:30.494762+0000","last_peered":"2024-09-11T05:14:30.494762+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:30.494762+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4174,"log_dups_size":0,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17080320,"num_objects":4170,"num_object_clones":0,"num_object_copies":8340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4324","reported_seq":4390,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:30.443754+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:30.443754+0000","last_peered":"2024-09-11T05:14:30.443754+0000","last_clean":"2024-09-11T05:14:30.443754+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:30.443754+0000","last_undegraded":"2024-09-11T05:14:30.443754+0000","last_fullsized":"2024-09-11T05:14:30.443754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4324,"log_dups_size":0,"ondisk_log_size":4324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17711104,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4324,"num_write_kb":17296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4212","reported_seq":4336,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:28.644901+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:28.644901+0000","last_peered":"2024-09-11T05:14:28.644901+0000","last_clean":"2024-09-11T05:14:28.644901+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:28.644901+0000","last_undegraded":"2024-09-11T05:14:28.644901+0000","last_fullsized":"2024-09-11T05:14:28.644901+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17223680,"num_objects":4205,"num_object_clones":0,"num_object_copies":8410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1046,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4148","reported_seq":4292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.617028+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:28.617028+0000","last_peered":"2024-09-11T05:14:28.617028+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:28.617028+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4148,"log_dups_size":0,"ondisk_log_size":4148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16969728,"num_objects":4143,"num_object_clones":0,"num_object_copies":8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":692,"num_write_kb":2768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":543},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4151","reported_seq":4278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.634197+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:28.634197+0000","last_peered":"2024-09-11T05:14:28.634197+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:28.634197+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4151,"log_dups_size":0,"ondisk_log_size":4151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1316,"num_write_kb":5264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4070","reported_seq":4188,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:28.627585+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:28.627585+0000","last_peered":"2024-09-11T05:14:28.627585+0000","last_clean":"2024-09-11T05:14:28.627585+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:28.627585+0000","last_undegraded":"2024-09-11T05:14:28.627585+0000","last_fullsized":"2024-09-11T05:14:28.627585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16662528,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1373,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4199","reported_seq":4325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:30.499935+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:30.499935+0000","last_peered":"2024-09-11T05:14:30.499935+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:30.499935+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4199,"log_dups_size":0,"ondisk_log_size":4199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2267,"num_write_kb":9068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4012","reported_seq":4172,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.590769+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:28.590769+0000","last_peered":"2024-09-11T05:14:28.590769+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:28.590769+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4012,"log_dups_size":0,"ondisk_log_size":4012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":765,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":562},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4150","reported_seq":4262,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:30.447651+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:30.447651+0000","last_peered":"2024-09-11T05:14:30.447651+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:30.447651+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4150,"log_dups_size":0,"ondisk_log_size":4150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":8716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":623}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4160","reported_seq":5045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:30.524690+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:30.524690+0000","last_peered":"2024-09-11T05:14:30.524690+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:30.524690+0000","last_undegraded":"2024-09-11T05:14:30.524690+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4160,"log_dups_size":0,"ondisk_log_size":4160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17014784,"num_objects":4154,"num_object_clones":0,"num_object_copies":8308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3092,"num_objects_unfound":0,"num_objects_dirty":4154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2063,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3092},{"shards":"2","objects":1062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4231","reported_seq":4356,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.659447+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:28.659447+0000","last_peered":"2024-09-11T05:14:28.659447+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:28.659447+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17321984,"num_objects":4229,"num_object_clones":0,"num_object_copies":8458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1264,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":606},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4051","reported_seq":4179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.619414+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:28.619414+0000","last_peered":"2024-09-11T05:14:28.619414+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:28.619414+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":917,"num_write_kb":3668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":548},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4072","reported_seq":4202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.640222+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:28.640222+0000","last_peered":"2024-09-11T05:14:28.640222+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:28.640222+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4072,"log_dups_size":0,"ondisk_log_size":4072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16662528,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":830,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":524},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4100","reported_seq":4238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:28.636189+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:28.636189+0000","last_peered":"2024-09-11T05:14:28.636189+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:28.636189+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16745065,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1107,"num_write_kb":4428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":574},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":272048745,"num_objects":66419,"num_object_clones":0,"num_object_copies":132838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":3092,"num_objects_unfound":0,"num_objects_dirty":66419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29564,"num_write_kb":118256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":1585152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548745216,"data_stored":548658241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66500,"ondisk_log_size":66500,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396220,"kb_used_data":247916,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93975620,"statfs":{"total":96636764160,"available":96231034880,"internally_reserved":0,"allocated":253865984,"data_stored":253621525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151842475},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89812,"kb_used_data":31748,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94282028,"statfs":{"total":96636764160,"available":96544796672,"internally_reserved":0,"allocated":32509952,"data_stored":32184878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":59436715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215128,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398404,"kb_used_data":254324,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973436,"statfs":{"total":96636764160,"available":96228798464,"internally_reserved":0,"allocated":260427776,"data_stored":260179240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,6,3,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260644864,"data_stored":260641385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33857536,"data_stored":33777519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254242816,"data_stored":254239337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:34.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 61 20 69259 69239 4.43189 2.29297 0.0676958 0.0176076 2024-09-11T05:14:35.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 62 20 69876 69856 4.39929 2.41016 0.0308284 0.0177465 2024-09-11T05:14:36.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 63 20 70512 70492 4.3689 2.48438 0.0320256 0.0178717 2024-09-11T05:14:36.644 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:37.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 64 20 71121 71101 4.33781 2.37891 0.0272496 0.0180007 2024-09-11T05:14:37.572 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:37.575 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:37.632 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-11T05:14:37.377423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":284508796,"num_objects":69463,"num_object_clones":0,"num_object_copies":138926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":69463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32608,"num_write_kb":130426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":2113555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69545,"ondisk_log_size":69545,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":917624,"kb_used_data":560200,"kb_used_omap":12,"kb_used_meta":357363,"kb_avail":282197896,"statfs":{"total":289910292480,"available":288970645504,"internally_reserved":0,"allocated":573644800,"data_stored":572826731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":365939713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,8,10,11,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30728192,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7502,"num_write_kb":30008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002633"},"pg_stats":[{"pgid":"2.f","version":"43'4437","reported_seq":4510,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.671015+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:33.671015+0000","last_peered":"2024-09-11T05:14:33.671015+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:33.671015+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18173952,"num_objects":4437,"num_object_clones":0,"num_object_copies":8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4437,"num_write_kb":17748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":762},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4401","reported_seq":4498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:35.539535+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:35.539535+0000","last_peered":"2024-09-11T05:14:35.539535+0000","last_clean":"2024-09-11T05:14:35.539535+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:35.539535+0000","last_undegraded":"2024-09-11T05:14:35.539535+0000","last_fullsized":"2024-09-11T05:14:35.539535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":4401,"log_dups_size":0,"ondisk_log_size":4401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18001920,"num_objects":4395,"num_object_clones":0,"num_object_copies":8790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2799,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4352","reported_seq":4472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.574532+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:35.574532+0000","last_peered":"2024-09-11T05:14:35.574532+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:35.574532+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17809408,"num_objects":4348,"num_object_clones":0,"num_object_copies":8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2755,"num_write_kb":11020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":854}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4521","reported_seq":4587,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:35.547661+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:35.547661+0000","last_peered":"2024-09-11T05:14:35.547661+0000","last_clean":"2024-09-11T05:14:35.547661+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:35.547661+0000","last_undegraded":"2024-09-11T05:14:35.547661+0000","last_fullsized":"2024-09-11T05:14:35.547661+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4521,"num_write_kb":18084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4395","reported_seq":4519,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:33.692325+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:33.692325+0000","last_peered":"2024-09-11T05:14:33.692325+0000","last_clean":"2024-09-11T05:14:33.692325+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:33.692325+0000","last_undegraded":"2024-09-11T05:14:33.692325+0000","last_fullsized":"2024-09-11T05:14:33.692325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17973248,"num_objects":4388,"num_object_clones":0,"num_object_copies":8776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1229,"num_write_kb":4916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4363","reported_seq":4506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.671271+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:33.671271+0000","last_peered":"2024-09-11T05:14:33.671271+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:33.671271+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17850368,"num_objects":4358,"num_object_clones":0,"num_object_copies":8716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":907,"num_write_kb":3628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":758},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4348","reported_seq":4475,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.647047+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:33.647047+0000","last_peered":"2024-09-11T05:14:33.647047+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:33.647047+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4348,"log_dups_size":0,"ondisk_log_size":4348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17784832,"num_objects":4342,"num_object_clones":0,"num_object_copies":8684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1513,"num_write_kb":6052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":766},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4250","reported_seq":4369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:33.552055+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:33.552055+0000","last_peered":"2024-09-11T05:14:33.552055+0000","last_clean":"2024-09-11T05:14:33.552055+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:33.552055+0000","last_undegraded":"2024-09-11T05:14:33.552055+0000","last_fullsized":"2024-09-11T05:14:33.552055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17399808,"num_objects":4248,"num_object_clones":0,"num_object_copies":8496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1553,"num_write_kb":6212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4395","reported_seq":4521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.536313+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:35.536313+0000","last_peered":"2024-09-11T05:14:35.536313+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:35.536313+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17973248,"num_objects":4388,"num_object_clones":0,"num_object_copies":8776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":863}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4209","reported_seq":4368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.689824+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:33.689824+0000","last_peered":"2024-09-11T05:14:33.689824+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:33.689824+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4209,"log_dups_size":0,"ondisk_log_size":4209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17207296,"num_objects":4201,"num_object_clones":0,"num_object_copies":8402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":962,"num_write_kb":3848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":759},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4338","reported_seq":4450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.565166+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:35.565166+0000","last_peered":"2024-09-11T05:14:35.565166+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:35.565166+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4338,"log_dups_size":0,"ondisk_log_size":4338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17747968,"num_objects":4333,"num_object_clones":0,"num_object_copies":8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4334","reported_seq":5477,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:35.553475+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:35.553475+0000","last_peered":"2024-09-11T05:14:35.553475+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:35.553475+0000","last_undegraded":"2024-09-11T05:14:35.553475+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17727488,"num_objects":4328,"num_object_clones":0,"num_object_copies":8656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":4328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2237,"num_write_kb":8948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2963},{"shards":"2","objects":1365}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4409","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.630143+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:33.630143+0000","last_peered":"2024-09-11T05:14:33.630143+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:33.630143+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1442,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":784},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4239","reported_seq":4367,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.687909+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:33.687909+0000","last_peered":"2024-09-11T05:14:33.687909+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:33.687909+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17330176,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1105,"num_write_kb":4420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":736},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4248","reported_seq":4378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.554683+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:33.554683+0000","last_peered":"2024-09-11T05:14:33.554683+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:33.554683+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4248,"log_dups_size":0,"ondisk_log_size":4248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383424,"num_objects":4244,"num_object_clones":0,"num_object_copies":8488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":4024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4303","reported_seq":4441,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.671081+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:33.671081+0000","last_peered":"2024-09-11T05:14:33.671081+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:33.671081+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4303,"log_dups_size":0,"ondisk_log_size":4303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17576553,"num_objects":4292,"num_object_clones":0,"num_object_copies":8584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1310,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":777},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":284508777,"num_objects":69461,"num_object_clones":0,"num_object_copies":138922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":69461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32606,"num_write_kb":130424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":574136320,"data_stored":574049345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69542,"ondisk_log_size":69542,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389556,"kb_used_data":253732,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93982284,"statfs":{"total":96636764160,"available":96237858816,"internally_reserved":0,"allocated":259821568,"data_stored":259577109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128760,"kb_used_data":42920,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94243080,"statfs":{"total":96636764160,"available":96504913920,"internally_reserved":0,"allocated":43950080,"data_stored":43625006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215129,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399308,"kb_used_data":263548,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93972532,"statfs":{"total":96636764160,"available":96227872768,"internally_reserved":0,"allocated":269873152,"data_stored":269624616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":138997419},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,6,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270073856,"data_stored":270070377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43704320,"data_stored":43624303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260358144,"data_stored":260354665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:37.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:38.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 65 20 71708 71688 4.30636 2.29297 0.0197062 0.0181312 2024-09-11T05:14:38.597 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:38.597 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:38.672 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-11T05:14:37.377423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":284508796,"num_objects":69463,"num_object_clones":0,"num_object_copies":138926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":69463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32608,"num_write_kb":130426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":2113555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69545,"ondisk_log_size":69545,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":917624,"kb_used_data":560200,"kb_used_omap":12,"kb_used_meta":357363,"kb_avail":282197896,"statfs":{"total":289910292480,"available":288970645504,"internally_reserved":0,"allocated":573644800,"data_stored":572826731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":365939713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,8,10,11,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30728192,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7502,"num_write_kb":30008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002633"},"pg_stats":[{"pgid":"2.f","version":"43'4437","reported_seq":4510,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.671015+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:33.671015+0000","last_peered":"2024-09-11T05:14:33.671015+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:33.671015+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18173952,"num_objects":4437,"num_object_clones":0,"num_object_copies":8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4437,"num_write_kb":17748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":762},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4401","reported_seq":4498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:35.539535+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:35.539535+0000","last_peered":"2024-09-11T05:14:35.539535+0000","last_clean":"2024-09-11T05:14:35.539535+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:35.539535+0000","last_undegraded":"2024-09-11T05:14:35.539535+0000","last_fullsized":"2024-09-11T05:14:35.539535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":4401,"log_dups_size":0,"ondisk_log_size":4401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18001920,"num_objects":4395,"num_object_clones":0,"num_object_copies":8790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2799,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4352","reported_seq":4472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.574532+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:35.574532+0000","last_peered":"2024-09-11T05:14:35.574532+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:35.574532+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17809408,"num_objects":4348,"num_object_clones":0,"num_object_copies":8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2755,"num_write_kb":11020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":854}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4521","reported_seq":4587,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:35.547661+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:35.547661+0000","last_peered":"2024-09-11T05:14:35.547661+0000","last_clean":"2024-09-11T05:14:35.547661+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:35.547661+0000","last_undegraded":"2024-09-11T05:14:35.547661+0000","last_fullsized":"2024-09-11T05:14:35.547661+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4521,"num_write_kb":18084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4395","reported_seq":4519,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:33.692325+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:33.692325+0000","last_peered":"2024-09-11T05:14:33.692325+0000","last_clean":"2024-09-11T05:14:33.692325+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:33.692325+0000","last_undegraded":"2024-09-11T05:14:33.692325+0000","last_fullsized":"2024-09-11T05:14:33.692325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17973248,"num_objects":4388,"num_object_clones":0,"num_object_copies":8776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1229,"num_write_kb":4916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4363","reported_seq":4506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.671271+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:33.671271+0000","last_peered":"2024-09-11T05:14:33.671271+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:33.671271+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17850368,"num_objects":4358,"num_object_clones":0,"num_object_copies":8716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":907,"num_write_kb":3628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":758},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4348","reported_seq":4475,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.647047+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:33.647047+0000","last_peered":"2024-09-11T05:14:33.647047+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:33.647047+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4348,"log_dups_size":0,"ondisk_log_size":4348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17784832,"num_objects":4342,"num_object_clones":0,"num_object_copies":8684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1513,"num_write_kb":6052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":766},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4250","reported_seq":4369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:33.552055+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:33.552055+0000","last_peered":"2024-09-11T05:14:33.552055+0000","last_clean":"2024-09-11T05:14:33.552055+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:33.552055+0000","last_undegraded":"2024-09-11T05:14:33.552055+0000","last_fullsized":"2024-09-11T05:14:33.552055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17399808,"num_objects":4248,"num_object_clones":0,"num_object_copies":8496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1553,"num_write_kb":6212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4395","reported_seq":4521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.536313+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:35.536313+0000","last_peered":"2024-09-11T05:14:35.536313+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:35.536313+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17973248,"num_objects":4388,"num_object_clones":0,"num_object_copies":8776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":863}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4209","reported_seq":4368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.689824+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:33.689824+0000","last_peered":"2024-09-11T05:14:33.689824+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:33.689824+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4209,"log_dups_size":0,"ondisk_log_size":4209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17207296,"num_objects":4201,"num_object_clones":0,"num_object_copies":8402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":962,"num_write_kb":3848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":759},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4338","reported_seq":4450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.565166+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:35.565166+0000","last_peered":"2024-09-11T05:14:35.565166+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:35.565166+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4338,"log_dups_size":0,"ondisk_log_size":4338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17747968,"num_objects":4333,"num_object_clones":0,"num_object_copies":8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4334","reported_seq":5477,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:35.553475+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:35.553475+0000","last_peered":"2024-09-11T05:14:35.553475+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:35.553475+0000","last_undegraded":"2024-09-11T05:14:35.553475+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17727488,"num_objects":4328,"num_object_clones":0,"num_object_copies":8656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":4328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2237,"num_write_kb":8948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2963},{"shards":"2","objects":1365}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4409","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.630143+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:33.630143+0000","last_peered":"2024-09-11T05:14:33.630143+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:33.630143+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4409,"log_dups_size":0,"ondisk_log_size":4409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1442,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":784},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4239","reported_seq":4367,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.687909+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:33.687909+0000","last_peered":"2024-09-11T05:14:33.687909+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:33.687909+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17330176,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1105,"num_write_kb":4420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":736},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4248","reported_seq":4378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.554683+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:33.554683+0000","last_peered":"2024-09-11T05:14:33.554683+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:33.554683+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4248,"log_dups_size":0,"ondisk_log_size":4248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383424,"num_objects":4244,"num_object_clones":0,"num_object_copies":8488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1006,"num_write_kb":4024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4303","reported_seq":4441,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:33.671081+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:33.671081+0000","last_peered":"2024-09-11T05:14:33.671081+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:33.671081+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4303,"log_dups_size":0,"ondisk_log_size":4303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17576553,"num_objects":4292,"num_object_clones":0,"num_object_copies":8584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1310,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":777},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":284508777,"num_objects":69461,"num_object_clones":0,"num_object_copies":138922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":69461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32606,"num_write_kb":130424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":574136320,"data_stored":574049345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69542,"ondisk_log_size":69542,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389556,"kb_used_data":253732,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93982284,"statfs":{"total":96636764160,"available":96237858816,"internally_reserved":0,"allocated":259821568,"data_stored":259577109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128760,"kb_used_data":42920,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94243080,"statfs":{"total":96636764160,"available":96504913920,"internally_reserved":0,"allocated":43950080,"data_stored":43625006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215129,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399308,"kb_used_data":263548,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93972532,"statfs":{"total":96636764160,"available":96227872768,"internally_reserved":0,"allocated":269873152,"data_stored":269624616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":138997419},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,6,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270073856,"data_stored":270070377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43704320,"data_stored":43624303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260358144,"data_stored":260354665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:38.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:14:38.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4437", 'reported_seq': 4510, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.671015+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:14:33.671015+0000', 'last_peered': '2024-09-11T05:14:33.671015+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:14:33.671015+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 4437, 'log_dups_size': 0, 'ondisk_log_size': 4437, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18173952, 'num_objects': 4437, 'num_object_clones': 0, 'num_object_copies': 8874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4437, 'num_write_kb': 17748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 762}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:14:38.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4352", 'reported_seq': 4472, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:35.574532+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:14:35.574532+0000', 'last_peered': '2024-09-11T05:14:35.574532+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:14:35.574532+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 4352, 'log_dups_size': 0, 'ondisk_log_size': 4352, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17809408, 'num_objects': 4348, 'num_object_clones': 0, 'num_object_copies': 8696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4348, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2755, 'num_write_kb': 11020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 854}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:38.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:14:38.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4363", 'reported_seq': 4506, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.671271+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:14:33.671271+0000', 'last_peered': '2024-09-11T05:14:33.671271+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:14:33.671271+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 4363, 'log_dups_size': 0, 'ondisk_log_size': 4363, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17850368, 'num_objects': 4358, 'num_object_clones': 0, 'num_object_copies': 8716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 907, 'num_write_kb': 3628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 758}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4348", 'reported_seq': 4475, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.647047+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:14:33.647047+0000', 'last_peered': '2024-09-11T05:14:33.647047+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:14:33.647047+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 4348, 'log_dups_size': 0, 'ondisk_log_size': 4348, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17784832, 'num_objects': 4342, 'num_object_clones': 0, 'num_object_copies': 8684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1513, 'num_write_kb': 6052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 766}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4395", 'reported_seq': 4521, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:35.536313+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:14:35.536313+0000', 'last_peered': '2024-09-11T05:14:35.536313+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:14:35.536313+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 4395, 'log_dups_size': 0, 'ondisk_log_size': 4395, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17973248, 'num_objects': 4388, 'num_object_clones': 0, 'num_object_copies': 8776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2463, 'num_write_kb': 9852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 863}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4209", 'reported_seq': 4368, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.689824+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:14:33.689824+0000', 'last_peered': '2024-09-11T05:14:33.689824+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:14:33.689824+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 4209, 'log_dups_size': 0, 'ondisk_log_size': 4209, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17207296, 'num_objects': 4201, 'num_object_clones': 0, 'num_object_copies': 8402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 962, 'num_write_kb': 3848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 759}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4338", 'reported_seq': 4450, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:35.565166+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:14:35.565166+0000', 'last_peered': '2024-09-11T05:14:35.565166+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:14:35.565166+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 4338, 'log_dups_size': 0, 'ondisk_log_size': 4338, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17747968, 'num_objects': 4333, 'num_object_clones': 0, 'num_object_copies': 8666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2367, 'num_write_kb': 9468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 811}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:38.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4334", 'reported_seq': 5477, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:14:35.553475+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:14:35.553475+0000', 'last_peered': '2024-09-11T05:14:35.553475+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:14:35.553475+0000', 'last_undegraded': '2024-09-11T05:14:35.553475+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 4334, 'log_dups_size': 0, 'ondisk_log_size': 4334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17727488, 'num_objects': 4328, 'num_object_clones': 0, 'num_object_copies': 8656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2963, 'num_objects_unfound': 0, 'num_objects_dirty': 4328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2237, 'num_write_kb': 8948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 515, 'num_bytes_recovered': 2109440, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2963}, {'shards': '2', 'objects': 1365}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4409", 'reported_seq': 4535, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.630143+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:14:33.630143+0000', 'last_peered': '2024-09-11T05:14:33.630143+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:14:33.630143+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 4409, 'log_dups_size': 0, 'ondisk_log_size': 4409, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18051072, 'num_objects': 4407, 'num_object_clones': 0, 'num_object_copies': 8814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1442, 'num_write_kb': 5768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 784}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4239", 'reported_seq': 4367, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.687909+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:14:33.687909+0000', 'last_peered': '2024-09-11T05:14:33.687909+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:14:33.687909+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 4239, 'log_dups_size': 0, 'ondisk_log_size': 4239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17330176, 'num_objects': 4231, 'num_object_clones': 0, 'num_object_copies': 8462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1105, 'num_write_kb': 4420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 736}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4248", 'reported_seq': 4378, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.554683+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:14:33.554683+0000', 'last_peered': '2024-09-11T05:14:33.554683+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:14:33.554683+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 4248, 'log_dups_size': 0, 'ondisk_log_size': 4248, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17383424, 'num_objects': 4244, 'num_object_clones': 0, 'num_object_copies': 8488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4244, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1006, 'num_write_kb': 4024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 700}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:14:38.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4303", 'reported_seq': 4441, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:33.671081+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:14:33.671081+0000', 'last_peered': '2024-09-11T05:14:33.671081+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:14:33.671081+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 4303, 'log_dups_size': 0, 'ondisk_log_size': 4303, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17576553, 'num_objects': 4292, 'num_object_clones': 0, 'num_object_copies': 8584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4292, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1310, 'num_write_kb': 5240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 777}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:38.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:14:39.092 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 66 20 72312 72292 4.27687 2.35938 0.0594434 0.0182579 2024-09-11T05:14:39.564 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:39.564 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/138926 objects degraded (28.095%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":157,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":69463,"data_bytes":284508796,"bytes_used":939646976,"bytes_avail":288970645504,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":138926,"degraded_ratio":0.28094813065948776,"misplaced_objects":2963,"misplaced_total":138926,"misplaced_ratio":0.021327901184803419,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":129678,"recovering_keys_per_sec":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"write_bytes_sec":2560120,"read_op_per_sec":0,"write_op_per_sec":625},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:14:39.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:14:39.624 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:40.092 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 67 20 72878 72858 4.24604 2.21094 0.00992607 0.0183894 2024-09-11T05:14:40.390 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:40.390 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:40.457 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-11T05:14:39.378826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":292274812,"num_objects":71359,"num_object_clones":0,"num_object_copies":142718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":71359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34504,"num_write_kb":138010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":2113555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71441,"ondisk_log_size":71441,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":934748,"kb_used_data":569004,"kb_used_omap":12,"kb_used_meta":365683,"kb_avail":282180772,"statfs":{"total":289910292480,"available":288953110528,"internally_reserved":0,"allocated":582660096,"data_stored":581842027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":374459393},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,3,8,10,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33312768,"num_objects":8133,"num_object_clones":0,"num_object_copies":16266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":8133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8133,"num_write_kb":32532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003559"},"pg_stats":[{"pgid":"2.f","version":"43'4628","reported_seq":4700,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.714424+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:38.714424+0000","last_peered":"2024-09-11T05:14:38.714424+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:38.714424+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4628,"num_write_kb":18512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":953},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4401","reported_seq":4498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:35.539535+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:35.539535+0000","last_peered":"2024-09-11T05:14:35.539535+0000","last_clean":"2024-09-11T05:14:35.539535+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:35.539535+0000","last_undegraded":"2024-09-11T05:14:35.539535+0000","last_fullsized":"2024-09-11T05:14:35.539535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":4401,"log_dups_size":0,"ondisk_log_size":4401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18001920,"num_objects":4395,"num_object_clones":0,"num_object_copies":8790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2799,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4352","reported_seq":4472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.574532+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:35.574532+0000","last_peered":"2024-09-11T05:14:35.574532+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:35.574532+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17809408,"num_objects":4348,"num_object_clones":0,"num_object_copies":8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2755,"num_write_kb":11020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":854}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4521","reported_seq":4587,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:35.547661+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:35.547661+0000","last_peered":"2024-09-11T05:14:35.547661+0000","last_clean":"2024-09-11T05:14:35.547661+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:35.547661+0000","last_undegraded":"2024-09-11T05:14:35.547661+0000","last_fullsized":"2024-09-11T05:14:35.547661+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4521,"num_write_kb":18084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4580","reported_seq":4704,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:38.682374+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:38.682374+0000","last_peered":"2024-09-11T05:14:38.682374+0000","last_clean":"2024-09-11T05:14:38.682374+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:38.682374+0000","last_undegraded":"2024-09-11T05:14:38.682374+0000","last_fullsized":"2024-09-11T05:14:38.682374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4580,"log_dups_size":0,"ondisk_log_size":4580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1414,"num_write_kb":5656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4565","reported_seq":4708,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.693444+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:38.693444+0000","last_peered":"2024-09-11T05:14:38.693444+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:38.693444+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4565,"log_dups_size":0,"ondisk_log_size":4565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1109,"num_write_kb":4436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":960},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4527","reported_seq":4654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.626214+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:38.626214+0000","last_peered":"2024-09-11T05:14:38.626214+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:38.626214+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4527,"log_dups_size":0,"ondisk_log_size":4527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1692,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4454","reported_seq":4573,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:38.589522+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:38.589522+0000","last_peered":"2024-09-11T05:14:38.589522+0000","last_clean":"2024-09-11T05:14:38.589522+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:38.589522+0000","last_undegraded":"2024-09-11T05:14:38.589522+0000","last_fullsized":"2024-09-11T05:14:38.589522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4454,"log_dups_size":0,"ondisk_log_size":4454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18235392,"num_objects":4452,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1757,"num_write_kb":7028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4395","reported_seq":4521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.536313+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:35.536313+0000","last_peered":"2024-09-11T05:14:35.536313+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:35.536313+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17973248,"num_objects":4388,"num_object_clones":0,"num_object_copies":8776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":863}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4385","reported_seq":4544,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.671678+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:38.671678+0000","last_peered":"2024-09-11T05:14:38.671678+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:38.671678+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17928192,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1138,"num_write_kb":4552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4338","reported_seq":4450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:35.565166+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:35.565166+0000","last_peered":"2024-09-11T05:14:35.565166+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:35.565166+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4338,"log_dups_size":0,"ondisk_log_size":4338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17747968,"num_objects":4333,"num_object_clones":0,"num_object_copies":8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2367,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4334","reported_seq":5477,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:35.553475+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:35.553475+0000","last_peered":"2024-09-11T05:14:35.553475+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:35.553475+0000","last_undegraded":"2024-09-11T05:14:35.553475+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17727488,"num_objects":4328,"num_object_clones":0,"num_object_copies":8656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":4328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2237,"num_write_kb":8948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2963},{"shards":"2","objects":1365}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4618","reported_seq":4744,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.650684+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:38.650684+0000","last_peered":"2024-09-11T05:14:38.650684+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:38.650684+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18907136,"num_objects":4616,"num_object_clones":0,"num_object_copies":9232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1651,"num_write_kb":6604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":993},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4438","reported_seq":4566,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.668478+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:38.668478+0000","last_peered":"2024-09-11T05:14:38.668478+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:38.668478+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4438,"log_dups_size":0,"ondisk_log_size":4438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18145280,"num_objects":4430,"num_object_clones":0,"num_object_copies":8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1304,"num_write_kb":5216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4415","reported_seq":4545,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.650571+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:38.650571+0000","last_peered":"2024-09-11T05:14:38.650571+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:38.650571+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4415,"log_dups_size":0,"ondisk_log_size":4415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1173,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":867},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4487","reported_seq":4625,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.657246+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:38.657246+0000","last_peered":"2024-09-11T05:14:38.657246+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:38.657246+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4487,"log_dups_size":0,"ondisk_log_size":4487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18330217,"num_objects":4476,"num_object_clones":0,"num_object_copies":8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1494,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":961},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":292274793,"num_objects":71357,"num_object_clones":0,"num_object_copies":142714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2963,"num_objects_unfound":0,"num_objects_dirty":71357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34502,"num_write_kb":138008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":583434240,"data_stored":583347265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71438,"ondisk_log_size":71438,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389556,"kb_used_data":253732,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93982284,"statfs":{"total":96636764160,"available":96237858816,"internally_reserved":0,"allocated":259821568,"data_stored":259577109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128760,"kb_used_data":42920,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94243080,"statfs":{"total":96636764160,"available":96504913920,"internally_reserved":0,"allocated":43950080,"data_stored":43625006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215130,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416432,"kb_used_data":272352,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93955408,"statfs":{"total":96636764160,"available":96210337792,"internally_reserved":0,"allocated":278888448,"data_stored":278639912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,3,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279371776,"data_stored":279368297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43704320,"data_stored":43624303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260358144,"data_stored":260354665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:41.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 68 20 73483 73463 4.21831 2.36328 0.00463946 0.0185116 2024-09-11T05:14:42.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 69 20 74106 74086 4.19245 2.43359 0.0323414 0.0186255 2024-09-11T05:14:43.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 70 20 74727 74707 4.16722 2.42578 0.0188444 0.0187356 2024-09-11T05:14:43.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:44.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 71 20 75323 75303 4.14132 2.32812 0.0158958 0.0188559 2024-09-11T05:14:44.427 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:44.496 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-11T05:14:43.379659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":296841852,"num_objects":72474,"num_object_clones":0,"num_object_copies":144948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2840,"num_objects_unfound":0,"num_objects_dirty":72474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35619,"num_write_kb":142470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":2617363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72556,"ondisk_log_size":72556,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":962592,"kb_used_data":584364,"kb_used_omap":12,"kb_used_meta":378163,"kb_avail":282152928,"statfs":{"total":289910292480,"available":288924598272,"internally_reserved":0,"allocated":598388736,"data_stored":597570667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":387238913},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,6,8,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6053,"num_write_kb":24212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"2.f","version":"43'4628","reported_seq":4700,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.714424+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:38.714424+0000","last_peered":"2024-09-11T05:14:38.714424+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:38.714424+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4628,"num_write_kb":18512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":953},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4599","reported_seq":4696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:40.542084+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:40.542084+0000","last_peered":"2024-09-11T05:14:40.542084+0000","last_clean":"2024-09-11T05:14:40.542084+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:40.542084+0000","last_undegraded":"2024-09-11T05:14:40.542084+0000","last_fullsized":"2024-09-11T05:14:40.542084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4531","reported_seq":4650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:40.585892+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:40.585892+0000","last_peered":"2024-09-11T05:14:40.585892+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:40.585892+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4531,"log_dups_size":0,"ondisk_log_size":4531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2934,"num_write_kb":11736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1033}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4673","reported_seq":4739,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:40.549045+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:40.549045+0000","last_peered":"2024-09-11T05:14:40.549045+0000","last_clean":"2024-09-11T05:14:40.549045+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:40.549045+0000","last_undegraded":"2024-09-11T05:14:40.549045+0000","last_fullsized":"2024-09-11T05:14:40.549045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4673,"log_dups_size":0,"ondisk_log_size":4673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4580","reported_seq":4704,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:38.682374+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:38.682374+0000","last_peered":"2024-09-11T05:14:38.682374+0000","last_clean":"2024-09-11T05:14:38.682374+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:38.682374+0000","last_undegraded":"2024-09-11T05:14:38.682374+0000","last_fullsized":"2024-09-11T05:14:38.682374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4580,"log_dups_size":0,"ondisk_log_size":4580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1414,"num_write_kb":5656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4565","reported_seq":4708,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.693444+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:38.693444+0000","last_peered":"2024-09-11T05:14:38.693444+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:38.693444+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4565,"log_dups_size":0,"ondisk_log_size":4565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1109,"num_write_kb":4436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":960},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4527","reported_seq":4654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.626214+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:38.626214+0000","last_peered":"2024-09-11T05:14:38.626214+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:38.626214+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4527,"log_dups_size":0,"ondisk_log_size":4527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1692,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4454","reported_seq":4573,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:38.589522+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:38.589522+0000","last_peered":"2024-09-11T05:14:38.589522+0000","last_clean":"2024-09-11T05:14:38.589522+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:38.589522+0000","last_undegraded":"2024-09-11T05:14:38.589522+0000","last_fullsized":"2024-09-11T05:14:38.589522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4454,"log_dups_size":0,"ondisk_log_size":4454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18235392,"num_objects":4452,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1757,"num_write_kb":7028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4590","reported_seq":4716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:40.510288+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:40.510288+0000","last_peered":"2024-09-11T05:14:40.510288+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:40.510288+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18771968,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2658,"num_write_kb":10632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4385","reported_seq":4544,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.671678+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:38.671678+0000","last_peered":"2024-09-11T05:14:38.671678+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:38.671678+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17928192,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1138,"num_write_kb":4552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4532","reported_seq":4644,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:40.511278+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:40.511278+0000","last_peered":"2024-09-11T05:14:40.511278+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:40.511278+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2561,"num_write_kb":10244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1005}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4531","reported_seq":5916,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:40.529522+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:40.529522+0000","last_peered":"2024-09-11T05:14:40.529522+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:40.529522+0000","last_undegraded":"2024-09-11T05:14:40.529522+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4531,"log_dups_size":0,"ondisk_log_size":4531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2840,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2434,"num_write_kb":9736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":2613248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2840},{"shards":"2","objects":1685}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4618","reported_seq":4744,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.650684+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:38.650684+0000","last_peered":"2024-09-11T05:14:38.650684+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:38.650684+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18907136,"num_objects":4616,"num_object_clones":0,"num_object_copies":9232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1651,"num_write_kb":6604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":993},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4438","reported_seq":4566,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.668478+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:38.668478+0000","last_peered":"2024-09-11T05:14:38.668478+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:38.668478+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4438,"log_dups_size":0,"ondisk_log_size":4438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18145280,"num_objects":4430,"num_object_clones":0,"num_object_copies":8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1304,"num_write_kb":5216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4415","reported_seq":4545,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.650571+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:38.650571+0000","last_peered":"2024-09-11T05:14:38.650571+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:38.650571+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4415,"log_dups_size":0,"ondisk_log_size":4415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1173,"num_write_kb":4692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":867},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4487","reported_seq":4625,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:38.657246+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:38.657246+0000","last_peered":"2024-09-11T05:14:38.657246+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:38.657246+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4487,"log_dups_size":0,"ondisk_log_size":4487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18330217,"num_objects":4476,"num_object_clones":0,"num_object_copies":8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1494,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":961},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":296841833,"num_objects":72472,"num_object_clones":0,"num_object_copies":144944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2840,"num_objects_unfound":0,"num_objects_dirty":72472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35617,"num_write_kb":142468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":639,"num_bytes_recovered":2617344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":599306240,"data_stored":599219265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72553,"ondisk_log_size":72553,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399800,"kb_used_data":259816,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93972040,"statfs":{"total":96636764160,"available":96227368960,"internally_reserved":0,"allocated":266051584,"data_stored":265807125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,5,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146360,"kb_used_data":52196,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94225480,"statfs":{"total":96636764160,"available":96486891520,"internally_reserved":0,"allocated":53448704,"data_stored":53123630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215130,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416432,"kb_used_data":272352,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93955408,"statfs":{"total":96636764160,"available":96210337792,"internally_reserved":0,"allocated":278888448,"data_stored":278639912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,3,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279371776,"data_stored":279368297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53444608,"data_stored":53364591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266489856,"data_stored":266486377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:44.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:45.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 72 20 75921 75901 4.11625 2.33594 0.0146275 0.0189698 2024-09-11T05:14:45.487 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:45.487 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:45.533 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-11T05:14:45.380148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":304652924,"num_objects":74381,"num_object_clones":0,"num_object_copies":148762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2840,"num_objects_unfound":0,"num_objects_dirty":74381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37526,"num_write_kb":150098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":2617363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74463,"ondisk_log_size":74463,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":975492,"kb_used_data":593104,"kb_used_omap":12,"kb_used_meta":382323,"kb_avail":282140028,"statfs":{"total":289910292480,"available":288911388672,"internally_reserved":0,"allocated":607338496,"data_stored":606520427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":391498753},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,8,6,6,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7960,"num_write_kb":31840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003499"},"pg_stats":[{"pgid":"2.f","version":"43'4827","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.677224+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:43.677224+0000","last_peered":"2024-09-11T05:14:43.677224+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:43.677224+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4827,"log_dups_size":0,"ondisk_log_size":4827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19771392,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4827,"num_write_kb":19308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4599","reported_seq":4696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:40.542084+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:40.542084+0000","last_peered":"2024-09-11T05:14:40.542084+0000","last_clean":"2024-09-11T05:14:40.542084+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:40.542084+0000","last_undegraded":"2024-09-11T05:14:40.542084+0000","last_fullsized":"2024-09-11T05:14:40.542084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4531","reported_seq":4650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:40.585892+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:40.585892+0000","last_peered":"2024-09-11T05:14:40.585892+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:40.585892+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4531,"log_dups_size":0,"ondisk_log_size":4531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2934,"num_write_kb":11736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1033}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4673","reported_seq":4739,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:40.549045+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:40.549045+0000","last_peered":"2024-09-11T05:14:40.549045+0000","last_clean":"2024-09-11T05:14:40.549045+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:40.549045+0000","last_undegraded":"2024-09-11T05:14:40.549045+0000","last_fullsized":"2024-09-11T05:14:40.549045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4673,"log_dups_size":0,"ondisk_log_size":4673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4765","reported_seq":4887,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:43.661356+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:43.661356+0000","last_peered":"2024-09-11T05:14:43.661356+0000","last_clean":"2024-09-11T05:14:43.661356+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:43.661356+0000","last_undegraded":"2024-09-11T05:14:43.661356+0000","last_fullsized":"2024-09-11T05:14:43.661356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4765,"log_dups_size":0,"ondisk_log_size":4765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1599,"num_write_kb":6396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4722","reported_seq":4866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.659983+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:43.659983+0000","last_peered":"2024-09-11T05:14:43.659983+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:43.659983+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4722,"log_dups_size":0,"ondisk_log_size":4722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19320832,"num_objects":4717,"num_object_clones":0,"num_object_copies":9434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1266,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1117},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4710","reported_seq":4837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.696451+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:43.696451+0000","last_peered":"2024-09-11T05:14:43.696451+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:43.696451+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1875,"num_write_kb":7500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1128},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4646","reported_seq":4765,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:43.581456+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:43.581456+0000","last_peered":"2024-09-11T05:14:43.581456+0000","last_clean":"2024-09-11T05:14:43.581456+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:43.581456+0000","last_undegraded":"2024-09-11T05:14:43.581456+0000","last_fullsized":"2024-09-11T05:14:43.581456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4646,"log_dups_size":0,"ondisk_log_size":4646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1949,"num_write_kb":7796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4590","reported_seq":4716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:40.510288+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:40.510288+0000","last_peered":"2024-09-11T05:14:40.510288+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:40.510288+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18771968,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2658,"num_write_kb":10632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4594","reported_seq":4754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.724451+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:43.724451+0000","last_peered":"2024-09-11T05:14:43.724451+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:43.724451+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4594,"log_dups_size":0,"ondisk_log_size":4594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18784256,"num_objects":4586,"num_object_clones":0,"num_object_copies":9172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1347,"num_write_kb":5388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1144},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4532","reported_seq":4644,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:40.511278+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:40.511278+0000","last_peered":"2024-09-11T05:14:40.511278+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:40.511278+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18542592,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2561,"num_write_kb":10244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1005}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4531","reported_seq":5916,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:40.529522+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:40.529522+0000","last_peered":"2024-09-11T05:14:40.529522+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:40.529522+0000","last_undegraded":"2024-09-11T05:14:40.529522+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4531,"log_dups_size":0,"ondisk_log_size":4531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2840,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2434,"num_write_kb":9736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":2613248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2840},{"shards":"2","objects":1685}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4827","reported_seq":4953,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.741761+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:43.741761+0000","last_peered":"2024-09-11T05:14:43.741761+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:43.741761+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4827,"log_dups_size":0,"ondisk_log_size":4827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1860,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1202},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4617","reported_seq":4745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.733419+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:43.733419+0000","last_peered":"2024-09-11T05:14:43.733419+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:43.733419+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18878464,"num_objects":4609,"num_object_clones":0,"num_object_copies":9218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1483,"num_write_kb":5932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1114},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4624","reported_seq":4754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.751800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:43.751800+0000","last_peered":"2024-09-11T05:14:43.751800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:43.751800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4624,"log_dups_size":0,"ondisk_log_size":4624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18923520,"num_objects":4620,"num_object_clones":0,"num_object_copies":9240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1382,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1076},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4672","reported_seq":4810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.705507+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:43.705507+0000","last_peered":"2024-09-11T05:14:43.705507+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:43.705507+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4672,"log_dups_size":0,"ondisk_log_size":4672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19087977,"num_objects":4661,"num_object_clones":0,"num_object_copies":9322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1679,"num_write_kb":6716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1146},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":304652905,"num_objects":74379,"num_object_clones":0,"num_object_copies":148758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2840,"num_objects_unfound":0,"num_objects_dirty":74379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37524,"num_write_kb":150096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":639,"num_bytes_recovered":2617344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":608690176,"data_stored":608603201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74460,"ondisk_log_size":74460,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399800,"kb_used_data":259816,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93972040,"statfs":{"total":96636764160,"available":96227368960,"internally_reserved":0,"allocated":266051584,"data_stored":265807125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,5,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146360,"kb_used_data":52196,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94225480,"statfs":{"total":96636764160,"available":96486891520,"internally_reserved":0,"allocated":53448704,"data_stored":53123630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215131,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429332,"kb_used_data":281092,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93942508,"statfs":{"total":96636764160,"available":96197128192,"internally_reserved":0,"allocated":287838208,"data_stored":287589672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288755712,"data_stored":288752233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53444608,"data_stored":53364591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266489856,"data_stored":266486377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:45.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:14:45.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4827", 'reported_seq': 4900, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.677224+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:14:43.677224+0000', 'last_peered': '2024-09-11T05:14:43.677224+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:14:43.677224+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 4827, 'log_dups_size': 0, 'ondisk_log_size': 4827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19771392, 'num_objects': 4827, 'num_object_clones': 0, 'num_object_copies': 9654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4827, 'num_write_kb': 19308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1152}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:14:45.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4531", 'reported_seq': 4650, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:40.585892+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:14:40.585892+0000', 'last_peered': '2024-09-11T05:14:40.585892+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:14:40.585892+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 4531, 'log_dups_size': 0, 'ondisk_log_size': 4531, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18542592, 'num_objects': 4527, 'num_object_clones': 0, 'num_object_copies': 9054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2934, 'num_write_kb': 11736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 1033}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:45.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:14:45.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4722", 'reported_seq': 4866, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.659983+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:14:43.659983+0000', 'last_peered': '2024-09-11T05:14:43.659983+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:14:43.659983+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 4722, 'log_dups_size': 0, 'ondisk_log_size': 4722, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19320832, 'num_objects': 4717, 'num_object_clones': 0, 'num_object_copies': 9434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1266, 'num_write_kb': 5064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1117}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4710", 'reported_seq': 4837, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.696451+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:14:43.696451+0000', 'last_peered': '2024-09-11T05:14:43.696451+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:14:43.696451+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 4710, 'log_dups_size': 0, 'ondisk_log_size': 4710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19267584, 'num_objects': 4704, 'num_object_clones': 0, 'num_object_copies': 9408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1875, 'num_write_kb': 7500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1128}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4590", 'reported_seq': 4716, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:40.510288+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:14:40.510288+0000', 'last_peered': '2024-09-11T05:14:40.510288+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:14:40.510288+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 4590, 'log_dups_size': 0, 'ondisk_log_size': 4590, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18771968, 'num_objects': 4583, 'num_object_clones': 0, 'num_object_copies': 9166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2658, 'num_write_kb': 10632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1058}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4594", 'reported_seq': 4754, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.724451+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:14:43.724451+0000', 'last_peered': '2024-09-11T05:14:43.724451+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:14:43.724451+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 4594, 'log_dups_size': 0, 'ondisk_log_size': 4594, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18784256, 'num_objects': 4586, 'num_object_clones': 0, 'num_object_copies': 9172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1347, 'num_write_kb': 5388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1144}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4532", 'reported_seq': 4644, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:40.511278+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:14:40.511278+0000', 'last_peered': '2024-09-11T05:14:40.511278+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:14:40.511278+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 4532, 'log_dups_size': 0, 'ondisk_log_size': 4532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18542592, 'num_objects': 4527, 'num_object_clones': 0, 'num_object_copies': 9054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2561, 'num_write_kb': 10244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 1005}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:14:45.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4531", 'reported_seq': 5916, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:14:40.529522+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:14:40.529522+0000', 'last_peered': '2024-09-11T05:14:40.529522+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:14:40.529522+0000', 'last_undegraded': '2024-09-11T05:14:40.529522+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 4531, 'log_dups_size': 0, 'ondisk_log_size': 4531, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18534400, 'num_objects': 4525, 'num_object_clones': 0, 'num_object_copies': 9050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2840, 'num_objects_unfound': 0, 'num_objects_dirty': 4525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2434, 'num_write_kb': 9736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 638, 'num_bytes_recovered': 2613248, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2840}, {'shards': '2', 'objects': 1685}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4827", 'reported_seq': 4953, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.741761+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:14:43.741761+0000', 'last_peered': '2024-09-11T05:14:43.741761+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:14:43.741761+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 4827, 'log_dups_size': 0, 'ondisk_log_size': 4827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19763200, 'num_objects': 4825, 'num_object_clones': 0, 'num_object_copies': 9650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4825, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1860, 'num_write_kb': 7440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1202}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4617", 'reported_seq': 4745, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.733419+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:14:43.733419+0000', 'last_peered': '2024-09-11T05:14:43.733419+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:14:43.733419+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 4617, 'log_dups_size': 0, 'ondisk_log_size': 4617, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18878464, 'num_objects': 4609, 'num_object_clones': 0, 'num_object_copies': 9218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4609, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1483, 'num_write_kb': 5932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1114}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4624", 'reported_seq': 4754, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.751800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:14:43.751800+0000', 'last_peered': '2024-09-11T05:14:43.751800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:14:43.751800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 4624, 'log_dups_size': 0, 'ondisk_log_size': 4624, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18923520, 'num_objects': 4620, 'num_object_clones': 0, 'num_object_copies': 9240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1382, 'num_write_kb': 5528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1076}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:14:45.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4672", 'reported_seq': 4810, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:43.705507+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:14:43.705507+0000', 'last_peered': '2024-09-11T05:14:43.705507+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:14:43.705507+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 4672, 'log_dups_size': 0, 'ondisk_log_size': 4672, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19087977, 'num_objects': 4661, 'num_object_clones': 0, 'num_object_copies': 9322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1679, 'num_write_kb': 6716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1146}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:45.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:14:46.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 73 20 76492 76472 4.09042 2.23047 0.0499582 0.0190908 2024-09-11T05:14:46.750 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:46.750 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/148762 objects degraded (26.237%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":164,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74381,"data_bytes":304652924,"bytes_used":998903808,"bytes_avail":288911388672,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":148762,"degraded_ratio":0.26237211115741921,"misplaced_objects":2840,"misplaced_total":148762,"misplaced_ratio":0.019090896868824026,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":85990,"recovering_keys_per_sec":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"write_bytes_sec":2716221,"read_op_per_sec":0,"write_op_per_sec":663},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:14:46.831 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:14:46.831 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:47.093 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 74 20 77070 77050 4.06566 2.25781 0.0202621 0.0192062 2024-09-11T05:14:47.607 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:47.608 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:47.655 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-11T05:14:47.380677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":309432956,"num_objects":75548,"num_object_clones":0,"num_object_copies":151096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2706,"num_objects_unfound":0,"num_objects_dirty":75548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38693,"num_write_kb":154766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":774,"num_bytes_recovered":3166227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75630,"ondisk_log_size":75630,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":986964,"kb_used_data":608740,"kb_used_omap":12,"kb_used_meta":378163,"kb_avail":282128556,"statfs":{"total":289910292480,"available":288899641344,"internally_reserved":0,"allocated":623349760,"data_stored":622531691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":387238913},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,5,4,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-386,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7207,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003618"},"pg_stats":[{"pgid":"2.f","version":"43'4827","reported_seq":4900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.677224+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:43.677224+0000","last_peered":"2024-09-11T05:14:43.677224+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:43.677224+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4827,"log_dups_size":0,"ondisk_log_size":4827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19771392,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4827,"num_write_kb":19308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4806","reported_seq":4903,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:45.602987+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:45.602987+0000","last_peered":"2024-09-11T05:14:45.602987+0000","last_clean":"2024-09-11T05:14:45.602987+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:45.602987+0000","last_undegraded":"2024-09-11T05:14:45.602987+0000","last_fullsized":"2024-09-11T05:14:45.602987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":4806,"log_dups_size":0,"ondisk_log_size":4806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19660800,"num_objects":4800,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3204,"num_write_kb":12816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4712","reported_seq":4832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:45.556515+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:45.556515+0000","last_peered":"2024-09-11T05:14:45.556515+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:45.556515+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19283968,"num_objects":4708,"num_object_clones":0,"num_object_copies":9416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3115,"num_write_kb":12460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4856","reported_seq":4922,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:45.592145+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:45.592145+0000","last_peered":"2024-09-11T05:14:45.592145+0000","last_clean":"2024-09-11T05:14:45.592145+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:45.592145+0000","last_undegraded":"2024-09-11T05:14:45.592145+0000","last_fullsized":"2024-09-11T05:14:45.592145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19890176,"num_objects":4856,"num_object_clones":0,"num_object_copies":9712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4856,"num_write_kb":19424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4765","reported_seq":4887,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:43.661356+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:43.661356+0000","last_peered":"2024-09-11T05:14:43.661356+0000","last_clean":"2024-09-11T05:14:43.661356+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:43.661356+0000","last_undegraded":"2024-09-11T05:14:43.661356+0000","last_fullsized":"2024-09-11T05:14:43.661356+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4765,"log_dups_size":0,"ondisk_log_size":4765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1599,"num_write_kb":6396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4722","reported_seq":4866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.659983+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:43.659983+0000","last_peered":"2024-09-11T05:14:43.659983+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:43.659983+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4722,"log_dups_size":0,"ondisk_log_size":4722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19320832,"num_objects":4717,"num_object_clones":0,"num_object_copies":9434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1266,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1117},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4710","reported_seq":4837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.696451+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:43.696451+0000","last_peered":"2024-09-11T05:14:43.696451+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:43.696451+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1875,"num_write_kb":7500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1128},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4646","reported_seq":4765,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:43.581456+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:43.581456+0000","last_peered":"2024-09-11T05:14:43.581456+0000","last_clean":"2024-09-11T05:14:43.581456+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:43.581456+0000","last_undegraded":"2024-09-11T05:14:43.581456+0000","last_fullsized":"2024-09-11T05:14:43.581456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4646,"log_dups_size":0,"ondisk_log_size":4646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1949,"num_write_kb":7796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4785","reported_seq":4910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:45.627972+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:45.627972+0000","last_peered":"2024-09-11T05:14:45.627972+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:45.627972+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570688,"num_objects":4778,"num_object_clones":0,"num_object_copies":9556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2853,"num_write_kb":11412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4594","reported_seq":4754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.724451+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:43.724451+0000","last_peered":"2024-09-11T05:14:43.724451+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:43.724451+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4594,"log_dups_size":0,"ondisk_log_size":4594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18784256,"num_objects":4586,"num_object_clones":0,"num_object_copies":9172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1347,"num_write_kb":5388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1144},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4744","reported_seq":4856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:45.587736+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:45.587736+0000","last_peered":"2024-09-11T05:14:45.587736+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:45.587736+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4744,"log_dups_size":0,"ondisk_log_size":4744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19410944,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4720","reported_seq":6374,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:45.579923+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:45.579923+0000","last_peered":"2024-09-11T05:14:45.579923+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:45.579923+0000","last_undegraded":"2024-09-11T05:14:45.579923+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4720,"log_dups_size":0,"ondisk_log_size":4720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19308544,"num_objects":4714,"num_object_clones":0,"num_object_copies":9428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2706,"num_objects_unfound":0,"num_objects_dirty":4714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2623,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":772,"num_bytes_recovered":3162112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2706},{"shards":"2","objects":2008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4827","reported_seq":4953,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.741761+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:43.741761+0000","last_peered":"2024-09-11T05:14:43.741761+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:43.741761+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":4827,"log_dups_size":0,"ondisk_log_size":4827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1860,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1202},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4617","reported_seq":4745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.733419+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:43.733419+0000","last_peered":"2024-09-11T05:14:43.733419+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:43.733419+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18878464,"num_objects":4609,"num_object_clones":0,"num_object_copies":9218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1483,"num_write_kb":5932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1114},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4624","reported_seq":4754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.751800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:43.751800+0000","last_peered":"2024-09-11T05:14:43.751800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:43.751800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4624,"log_dups_size":0,"ondisk_log_size":4624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18923520,"num_objects":4620,"num_object_clones":0,"num_object_copies":9240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1382,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1076},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4672","reported_seq":4810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:43.705507+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:43.705507+0000","last_peered":"2024-09-11T05:14:43.705507+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:43.705507+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4672,"log_dups_size":0,"ondisk_log_size":4672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19087977,"num_objects":4661,"num_object_clones":0,"num_object_copies":9322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1679,"num_write_kb":6716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1146},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":309432937,"num_objects":75546,"num_object_clones":0,"num_object_copies":151092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2706,"num_objects_unfound":0,"num_objects_dirty":75546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38691,"num_write_kb":154764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":3166208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":624971776,"data_stored":624884801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75627,"ondisk_log_size":75627,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409984,"kb_used_data":265840,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93961856,"statfs":{"total":96636764160,"available":96216940544,"internally_reserved":0,"allocated":272220160,"data_stored":271975701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147582635},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147648,"kb_used_data":61808,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94224192,"statfs":{"total":96636764160,"available":96485572608,"internally_reserved":0,"allocated":63291392,"data_stored":62966318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215131,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429332,"kb_used_data":281092,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93942508,"statfs":{"total":96636764160,"available":96197128192,"internally_reserved":0,"allocated":287838208,"data_stored":287589672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288755712,"data_stored":288752233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":63422464,"data_stored":63342447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272793600,"data_stored":272790121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:48.097 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 75 20 77653 77633 4.04182 2.27734 0.0311745 0.0193211 2024-09-11T05:14:49.094 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 76 20 78272 78252 4.02046 2.41797 0.0407265 0.0194239 2024-09-11T05:14:50.097 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 77 20 78893 78873 3.99955 2.42578 0.0101056 0.0195218 2024-09-11T05:14:50.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:51.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 78 20 79470 79450 3.97717 2.25391 0.0112792 0.019632 2024-09-11T05:14:51.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:51.575 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-11T05:14:51.383230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":321577596,"num_objects":78513,"num_object_clones":0,"num_object_copies":157026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":78513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41658,"num_write_kb":166626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3772435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78595,"ondisk_log_size":78595,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1020952,"kb_used_data":633000,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282094568,"statfs":{"total":289910292480,"available":288864837632,"internally_reserved":0,"allocated":648192000,"data_stored":647373931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":397200385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,2,2,9,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29302784,"num_objects":7154,"num_object_clones":0,"num_object_copies":14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-405,"num_objects_unfound":0,"num_objects_dirty":7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7154,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1658880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004404"},"pg_stats":[{"pgid":"2.f","version":"43'5008","reported_seq":5081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.726009+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:48.726009+0000","last_peered":"2024-09-11T05:14:48.726009+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:48.726009+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1333},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5003","reported_seq":5100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:50.632237+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:50.632237+0000","last_peered":"2024-09-11T05:14:50.632237+0000","last_clean":"2024-09-11T05:14:50.632237+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:50.632237+0000","last_undegraded":"2024-09-11T05:14:50.632237+0000","last_fullsized":"2024-09-11T05:14:50.632237+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20467712,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4897","reported_seq":5017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.616445+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:50.616445+0000","last_peered":"2024-09-11T05:14:50.616445+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:50.616445+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20041728,"num_objects":4893,"num_object_clones":0,"num_object_copies":9786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3300,"num_write_kb":13200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5045","reported_seq":5111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:50.643398+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:50.643398+0000","last_peered":"2024-09-11T05:14:50.643398+0000","last_clean":"2024-09-11T05:14:50.643398+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:50.643398+0000","last_undegraded":"2024-09-11T05:14:50.643398+0000","last_fullsized":"2024-09-11T05:14:50.643398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5045,"log_dups_size":0,"ondisk_log_size":5045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20664320,"num_objects":5045,"num_object_clones":0,"num_object_copies":10090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5045,"num_write_kb":20180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4949","reported_seq":5073,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:48.728440+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:48.728440+0000","last_peered":"2024-09-11T05:14:48.728440+0000","last_clean":"2024-09-11T05:14:48.728440+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:48.728440+0000","last_undegraded":"2024-09-11T05:14:48.728440+0000","last_fullsized":"2024-09-11T05:14:48.728440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4919","reported_seq":5059,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.773203+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:48.773203+0000","last_peered":"2024-09-11T05:14:48.773203+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:48.773203+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1463,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1314},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4897","reported_seq":5024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.783110+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:48.783110+0000","last_peered":"2024-09-11T05:14:48.783110+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:48.783110+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20033536,"num_objects":4891,"num_object_clones":0,"num_object_copies":9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2062,"num_write_kb":8248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1315},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4840","reported_seq":4959,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:48.618832+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:48.618832+0000","last_peered":"2024-09-11T05:14:48.618832+0000","last_clean":"2024-09-11T05:14:48.618832+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:48.618832+0000","last_undegraded":"2024-09-11T05:14:48.618832+0000","last_fullsized":"2024-09-11T05:14:48.618832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2143,"num_write_kb":8572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4966","reported_seq":5092,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.578167+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:50.578167+0000","last_peered":"2024-09-11T05:14:50.578167+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:50.578167+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4966,"log_dups_size":0,"ondisk_log_size":4966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20312064,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3034,"num_write_kb":12136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1434}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4767","reported_seq":4926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.761015+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:48.761015+0000","last_peered":"2024-09-11T05:14:48.761015+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:48.761015+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19492864,"num_objects":4759,"num_object_clones":0,"num_object_copies":9518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1520,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4953","reported_seq":5065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.648381+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:50.648381+0000","last_peered":"2024-09-11T05:14:50.648381+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:50.648381+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267008,"num_objects":4948,"num_object_clones":0,"num_object_copies":9896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1426}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4895","reported_seq":6848,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:50.607575+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:50.607575+0000","last_peered":"2024-09-11T05:14:50.607575+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:50.607575+0000","last_undegraded":"2024-09-11T05:14:50.607575+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2798,"num_write_kb":11192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":920,"num_bytes_recovered":3768320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":2331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5012","reported_seq":5135,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.782643+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:48.782643+0000","last_peered":"2024-09-11T05:14:48.782643+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:48.782643+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20520960,"num_objects":5010,"num_object_clones":0,"num_object_copies":10020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2045,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1387},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4798","reported_seq":4926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.649054+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:48.649054+0000","last_peered":"2024-09-11T05:14:48.649054+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:48.649054+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4798,"log_dups_size":0,"ondisk_log_size":4798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19619840,"num_objects":4790,"num_object_clones":0,"num_object_copies":9580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1664,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1295},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4800","reported_seq":4930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.713963+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:48.713963+0000","last_peered":"2024-09-11T05:14:48.713963+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:48.713963+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1558,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4843","reported_seq":4981,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.762571+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:48.762571+0000","last_peered":"2024-09-11T05:14:48.762571+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:48.762571+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19788393,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":7400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":321577577,"num_objects":78511,"num_object_clones":0,"num_object_copies":157022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":78511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41656,"num_write_kb":166624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":921,"num_bytes_recovered":3772416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":649953280,"data_stored":649866305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78592,"ondisk_log_size":78592,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407816,"kb_used_data":271992,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93964024,"statfs":{"total":96636764160,"available":96219160576,"internally_reserved":0,"allocated":278519808,"data_stored":278275349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":165308,"kb_used_data":71148,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94206532,"statfs":{"total":96636764160,"available":96467488768,"internally_reserved":0,"allocated":72855552,"data_stored":72530478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215132,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447828,"kb_used_data":289860,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93924012,"statfs":{"total":96636764160,"available":96178188288,"internally_reserved":0,"allocated":296816640,"data_stored":296568104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161738411},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,0,4,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297779200,"data_stored":297775721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73113600,"data_stored":73033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279060480,"data_stored":279057001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:51.577 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:52.097 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 79 20 80042 80022 3.95511 2.23438 0.0111838 0.0197406 2024-09-11T05:14:52.583 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:52.585 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:52.650 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-11T05:14:51.383230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":321577596,"num_objects":78513,"num_object_clones":0,"num_object_copies":157026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":78513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41658,"num_write_kb":166626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3772435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78595,"ondisk_log_size":78595,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1020952,"kb_used_data":633000,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282094568,"statfs":{"total":289910292480,"available":288864837632,"internally_reserved":0,"allocated":648192000,"data_stored":647373931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":397200385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,2,2,9,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29302784,"num_objects":7154,"num_object_clones":0,"num_object_copies":14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-405,"num_objects_unfound":0,"num_objects_dirty":7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7154,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1658880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004404"},"pg_stats":[{"pgid":"2.f","version":"43'5008","reported_seq":5081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.726009+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:48.726009+0000","last_peered":"2024-09-11T05:14:48.726009+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:48.726009+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1333},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5003","reported_seq":5100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:50.632237+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:50.632237+0000","last_peered":"2024-09-11T05:14:50.632237+0000","last_clean":"2024-09-11T05:14:50.632237+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:50.632237+0000","last_undegraded":"2024-09-11T05:14:50.632237+0000","last_fullsized":"2024-09-11T05:14:50.632237+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20467712,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4897","reported_seq":5017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.616445+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:50.616445+0000","last_peered":"2024-09-11T05:14:50.616445+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:50.616445+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20041728,"num_objects":4893,"num_object_clones":0,"num_object_copies":9786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3300,"num_write_kb":13200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5045","reported_seq":5111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:50.643398+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:50.643398+0000","last_peered":"2024-09-11T05:14:50.643398+0000","last_clean":"2024-09-11T05:14:50.643398+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:50.643398+0000","last_undegraded":"2024-09-11T05:14:50.643398+0000","last_fullsized":"2024-09-11T05:14:50.643398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5045,"log_dups_size":0,"ondisk_log_size":5045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20664320,"num_objects":5045,"num_object_clones":0,"num_object_copies":10090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5045,"num_write_kb":20180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4949","reported_seq":5073,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:48.728440+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:48.728440+0000","last_peered":"2024-09-11T05:14:48.728440+0000","last_clean":"2024-09-11T05:14:48.728440+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:48.728440+0000","last_undegraded":"2024-09-11T05:14:48.728440+0000","last_fullsized":"2024-09-11T05:14:48.728440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4919","reported_seq":5059,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.773203+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:48.773203+0000","last_peered":"2024-09-11T05:14:48.773203+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:48.773203+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1463,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1314},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4897","reported_seq":5024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.783110+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:48.783110+0000","last_peered":"2024-09-11T05:14:48.783110+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:48.783110+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20033536,"num_objects":4891,"num_object_clones":0,"num_object_copies":9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2062,"num_write_kb":8248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1315},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4840","reported_seq":4959,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:48.618832+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:48.618832+0000","last_peered":"2024-09-11T05:14:48.618832+0000","last_clean":"2024-09-11T05:14:48.618832+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:48.618832+0000","last_undegraded":"2024-09-11T05:14:48.618832+0000","last_fullsized":"2024-09-11T05:14:48.618832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2143,"num_write_kb":8572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4966","reported_seq":5092,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.578167+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:50.578167+0000","last_peered":"2024-09-11T05:14:50.578167+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:50.578167+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4966,"log_dups_size":0,"ondisk_log_size":4966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20312064,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3034,"num_write_kb":12136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1434}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4767","reported_seq":4926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.761015+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:48.761015+0000","last_peered":"2024-09-11T05:14:48.761015+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:48.761015+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19492864,"num_objects":4759,"num_object_clones":0,"num_object_copies":9518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1520,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4953","reported_seq":5065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.648381+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:50.648381+0000","last_peered":"2024-09-11T05:14:50.648381+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:50.648381+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267008,"num_objects":4948,"num_object_clones":0,"num_object_copies":9896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1426}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4895","reported_seq":6848,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:50.607575+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:50.607575+0000","last_peered":"2024-09-11T05:14:50.607575+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:50.607575+0000","last_undegraded":"2024-09-11T05:14:50.607575+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2798,"num_write_kb":11192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":920,"num_bytes_recovered":3768320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":2331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5012","reported_seq":5135,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.782643+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:48.782643+0000","last_peered":"2024-09-11T05:14:48.782643+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:48.782643+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20520960,"num_objects":5010,"num_object_clones":0,"num_object_copies":10020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2045,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1387},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4798","reported_seq":4926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.649054+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:48.649054+0000","last_peered":"2024-09-11T05:14:48.649054+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:48.649054+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4798,"log_dups_size":0,"ondisk_log_size":4798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19619840,"num_objects":4790,"num_object_clones":0,"num_object_copies":9580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1664,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1295},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4800","reported_seq":4930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.713963+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:48.713963+0000","last_peered":"2024-09-11T05:14:48.713963+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:48.713963+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1558,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4843","reported_seq":4981,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.762571+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:48.762571+0000","last_peered":"2024-09-11T05:14:48.762571+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:48.762571+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19788393,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":7400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":321577577,"num_objects":78511,"num_object_clones":0,"num_object_copies":157022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":78511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41656,"num_write_kb":166624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":921,"num_bytes_recovered":3772416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":649953280,"data_stored":649866305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78592,"ondisk_log_size":78592,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407816,"kb_used_data":271992,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93964024,"statfs":{"total":96636764160,"available":96219160576,"internally_reserved":0,"allocated":278519808,"data_stored":278275349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":165308,"kb_used_data":71148,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94206532,"statfs":{"total":96636764160,"available":96467488768,"internally_reserved":0,"allocated":72855552,"data_stored":72530478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215132,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447828,"kb_used_data":289860,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93924012,"statfs":{"total":96636764160,"available":96178188288,"internally_reserved":0,"allocated":296816640,"data_stored":296568104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161738411},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,0,4,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297779200,"data_stored":297775721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73113600,"data_stored":73033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279060480,"data_stored":279057001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:52.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5008", 'reported_seq': 5081, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.726009+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:14:48.726009+0000', 'last_peered': '2024-09-11T05:14:48.726009+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:14:48.726009+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 5008, 'log_dups_size': 0, 'ondisk_log_size': 5008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20512768, 'num_objects': 5008, 'num_object_clones': 0, 'num_object_copies': 10016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5008, 'num_write_kb': 20032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1333}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4897", 'reported_seq': 5017, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:50.616445+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:14:50.616445+0000', 'last_peered': '2024-09-11T05:14:50.616445+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:14:50.616445+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 4897, 'log_dups_size': 0, 'ondisk_log_size': 4897, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20041728, 'num_objects': 4893, 'num_object_clones': 0, 'num_object_copies': 9786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3300, 'num_write_kb': 13200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 1399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4919", 'reported_seq': 5059, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.773203+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:14:48.773203+0000', 'last_peered': '2024-09-11T05:14:48.773203+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:14:48.773203+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 4919, 'log_dups_size': 0, 'ondisk_log_size': 4919, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20127744, 'num_objects': 4914, 'num_object_clones': 0, 'num_object_copies': 9828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4914, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1463, 'num_write_kb': 5852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1314}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4897", 'reported_seq': 5024, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.783110+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:14:48.783110+0000', 'last_peered': '2024-09-11T05:14:48.783110+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:14:48.783110+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 4897, 'log_dups_size': 0, 'ondisk_log_size': 4897, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20033536, 'num_objects': 4891, 'num_object_clones': 0, 'num_object_copies': 9782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2062, 'num_write_kb': 8248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1315}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:14:52.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4966", 'reported_seq': 5092, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:50.578167+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:14:50.578167+0000', 'last_peered': '2024-09-11T05:14:50.578167+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:14:50.578167+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 4966, 'log_dups_size': 0, 'ondisk_log_size': 4966, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20312064, 'num_objects': 4959, 'num_object_clones': 0, 'num_object_copies': 9918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4959, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3034, 'num_write_kb': 12136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1434}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4767", 'reported_seq': 4926, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.761015+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:14:48.761015+0000', 'last_peered': '2024-09-11T05:14:48.761015+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:14:48.761015+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 4767, 'log_dups_size': 0, 'ondisk_log_size': 4767, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19492864, 'num_objects': 4759, 'num_object_clones': 0, 'num_object_copies': 9518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4759, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1520, 'num_write_kb': 6080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1317}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4953", 'reported_seq': 5065, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:50.648381+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:14:50.648381+0000', 'last_peered': '2024-09-11T05:14:50.648381+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:14:50.648381+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 4953, 'log_dups_size': 0, 'ondisk_log_size': 4953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20267008, 'num_objects': 4948, 'num_object_clones': 0, 'num_object_copies': 9896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2982, 'num_write_kb': 11928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 1426}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4895", 'reported_seq': 6848, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:14:50.607575+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:14:50.607575+0000', 'last_peered': '2024-09-11T05:14:50.607575+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:14:50.607575+0000', 'last_undegraded': '2024-09-11T05:14:50.607575+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 4895, 'log_dups_size': 0, 'ondisk_log_size': 4895, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20025344, 'num_objects': 4889, 'num_object_clones': 0, 'num_object_copies': 9778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2558, 'num_objects_unfound': 0, 'num_objects_dirty': 4889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2798, 'num_write_kb': 11192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 920, 'num_bytes_recovered': 3768320, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2558}, {'shards': '2', 'objects': 2331}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5012", 'reported_seq': 5135, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.782643+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:14:48.782643+0000', 'last_peered': '2024-09-11T05:14:48.782643+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:14:48.782643+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 5012, 'log_dups_size': 0, 'ondisk_log_size': 5012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20520960, 'num_objects': 5010, 'num_object_clones': 0, 'num_object_copies': 10020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2045, 'num_write_kb': 8180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1387}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:14:52.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4798", 'reported_seq': 4926, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.649054+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:14:48.649054+0000', 'last_peered': '2024-09-11T05:14:48.649054+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:14:48.649054+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 4798, 'log_dups_size': 0, 'ondisk_log_size': 4798, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19619840, 'num_objects': 4790, 'num_object_clones': 0, 'num_object_copies': 9580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1664, 'num_write_kb': 6656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1295}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:14:52.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4800", 'reported_seq': 4930, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.713963+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:14:48.713963+0000', 'last_peered': '2024-09-11T05:14:48.713963+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:14:48.713963+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 4800, 'log_dups_size': 0, 'ondisk_log_size': 4800, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19644416, 'num_objects': 4796, 'num_object_clones': 0, 'num_object_copies': 9592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1558, 'num_write_kb': 6232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1252}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:14:52.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4843", 'reported_seq': 4981, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:48.762571+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:14:48.762571+0000', 'last_peered': '2024-09-11T05:14:48.762571+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:14:48.762571+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 4843, 'log_dups_size': 0, 'ondisk_log_size': 4843, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19788393, 'num_objects': 4832, 'num_object_clones': 0, 'num_object_copies': 9664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1850, 'num_write_kb': 7400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1317}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:14:52.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:14:53.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:14:53.098851+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0198483 2024-09-11T05:14:53.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:14:53.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 80 20 80625 80605 3.93415 2.27734 0.0279639 0.0198483 2024-09-11T05:14:54.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 81 20 81227 81207 3.91461 2.35156 0.00954466 0.019946 2024-09-11T05:14:54.155 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:54.155 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/157026 objects degraded (24.856%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":171,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":78513,"data_bytes":321577596,"bytes_used":1045454848,"bytes_avail":288864837632,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":157026,"degraded_ratio":0.24856393208767971,"misplaced_objects":2558,"misplaced_total":157026,"misplaced_ratio":0.016290295874568542,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96219,"recovering_keys_per_sec":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"write_bytes_sec":2060523,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:14:54.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:14:54.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:55.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 82 20 81785 81765 3.89346 2.17969 0.00969107 0.0200552 2024-09-11T05:14:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:55.174 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:55.234 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-11T05:14:53.383776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":321577596,"num_objects":78513,"num_object_clones":0,"num_object_copies":157026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":78513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41658,"num_write_kb":166626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":3772435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78595,"ondisk_log_size":78595,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1020952,"kb_used_data":633000,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282094568,"statfs":{"total":289910292480,"available":288864837632,"internally_reserved":0,"allocated":648192000,"data_stored":647373931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":397200385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,2,2,9,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24735744,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6039,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004590"},"pg_stats":[{"pgid":"2.f","version":"43'5008","reported_seq":5081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.726009+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:48.726009+0000","last_peered":"2024-09-11T05:14:48.726009+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:48.726009+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1333},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5003","reported_seq":5100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:50.632237+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:50.632237+0000","last_peered":"2024-09-11T05:14:50.632237+0000","last_clean":"2024-09-11T05:14:50.632237+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:50.632237+0000","last_undegraded":"2024-09-11T05:14:50.632237+0000","last_fullsized":"2024-09-11T05:14:50.632237+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20467712,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4897","reported_seq":5017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.616445+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:50.616445+0000","last_peered":"2024-09-11T05:14:50.616445+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:50.616445+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20041728,"num_objects":4893,"num_object_clones":0,"num_object_copies":9786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3300,"num_write_kb":13200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5045","reported_seq":5111,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:50.643398+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:50.643398+0000","last_peered":"2024-09-11T05:14:50.643398+0000","last_clean":"2024-09-11T05:14:50.643398+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:50.643398+0000","last_undegraded":"2024-09-11T05:14:50.643398+0000","last_fullsized":"2024-09-11T05:14:50.643398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5045,"log_dups_size":0,"ondisk_log_size":5045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20664320,"num_objects":5045,"num_object_clones":0,"num_object_copies":10090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5045,"num_write_kb":20180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4949","reported_seq":5073,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:48.728440+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:48.728440+0000","last_peered":"2024-09-11T05:14:48.728440+0000","last_clean":"2024-09-11T05:14:48.728440+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:48.728440+0000","last_undegraded":"2024-09-11T05:14:48.728440+0000","last_fullsized":"2024-09-11T05:14:48.728440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4919","reported_seq":5059,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.773203+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:48.773203+0000","last_peered":"2024-09-11T05:14:48.773203+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:48.773203+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1463,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1314},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4897","reported_seq":5024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.783110+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:48.783110+0000","last_peered":"2024-09-11T05:14:48.783110+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:48.783110+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20033536,"num_objects":4891,"num_object_clones":0,"num_object_copies":9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2062,"num_write_kb":8248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1315},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4840","reported_seq":4959,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:48.618832+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:48.618832+0000","last_peered":"2024-09-11T05:14:48.618832+0000","last_clean":"2024-09-11T05:14:48.618832+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:48.618832+0000","last_undegraded":"2024-09-11T05:14:48.618832+0000","last_fullsized":"2024-09-11T05:14:48.618832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2143,"num_write_kb":8572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4966","reported_seq":5092,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.578167+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:50.578167+0000","last_peered":"2024-09-11T05:14:50.578167+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:50.578167+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":4966,"log_dups_size":0,"ondisk_log_size":4966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20312064,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3034,"num_write_kb":12136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1434}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4767","reported_seq":4926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.761015+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:48.761015+0000","last_peered":"2024-09-11T05:14:48.761015+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:48.761015+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19492864,"num_objects":4759,"num_object_clones":0,"num_object_copies":9518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1520,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4953","reported_seq":5065,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:50.648381+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:50.648381+0000","last_peered":"2024-09-11T05:14:50.648381+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:50.648381+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267008,"num_objects":4948,"num_object_clones":0,"num_object_copies":9896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1426}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4895","reported_seq":6848,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:50.607575+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:50.607575+0000","last_peered":"2024-09-11T05:14:50.607575+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:50.607575+0000","last_undegraded":"2024-09-11T05:14:50.607575+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2798,"num_write_kb":11192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":920,"num_bytes_recovered":3768320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":2331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5012","reported_seq":5135,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.782643+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:48.782643+0000","last_peered":"2024-09-11T05:14:48.782643+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:48.782643+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20520960,"num_objects":5010,"num_object_clones":0,"num_object_copies":10020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2045,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1387},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4798","reported_seq":4926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.649054+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:48.649054+0000","last_peered":"2024-09-11T05:14:48.649054+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:48.649054+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4798,"log_dups_size":0,"ondisk_log_size":4798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19619840,"num_objects":4790,"num_object_clones":0,"num_object_copies":9580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1664,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1295},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4800","reported_seq":4930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.713963+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:48.713963+0000","last_peered":"2024-09-11T05:14:48.713963+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:48.713963+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1558,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4843","reported_seq":4981,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:48.762571+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:48.762571+0000","last_peered":"2024-09-11T05:14:48.762571+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:48.762571+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19788393,"num_objects":4832,"num_object_clones":0,"num_object_copies":9664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":7400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1317},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":321577577,"num_objects":78511,"num_object_clones":0,"num_object_copies":157022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2558,"num_objects_unfound":0,"num_objects_dirty":78511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41656,"num_write_kb":166624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":921,"num_bytes_recovered":3772416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":649953280,"data_stored":649866305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78592,"ondisk_log_size":78592,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407816,"kb_used_data":271992,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93964024,"statfs":{"total":96636764160,"available":96219160576,"internally_reserved":0,"allocated":278519808,"data_stored":278275349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":139062955},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":165308,"kb_used_data":71148,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94206532,"statfs":{"total":96636764160,"available":96467488768,"internally_reserved":0,"allocated":72855552,"data_stored":72530478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":96399019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215132,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447828,"kb_used_data":289860,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93924012,"statfs":{"total":96636764160,"available":96178188288,"internally_reserved":0,"allocated":296816640,"data_stored":296568104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161738411},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,0,4,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297779200,"data_stored":297775721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73113600,"data_stored":73033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279060480,"data_stored":279057001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:56.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 83 20 82404 82384 3.87569 2.41797 0.0463283 0.0201488 2024-09-11T05:14:57.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 84 20 82997 82977 3.85712 2.31641 0.020332 0.0202473 2024-09-11T05:14:58.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 85 20 83592 83572 3.83909 2.32422 0.0254419 0.0203405 2024-09-11T05:14:58.236 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:14:59.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 86 20 84164 84144 3.82044 2.23438 0.0461488 0.0204411 2024-09-11T05:14:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:14:59.105 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:14:59.164 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-11T05:14:57.384752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":333533820,"num_objects":81432,"num_object_clones":0,"num_object_copies":162864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2422,"num_objects_unfound":0,"num_objects_dirty":81432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44577,"num_write_kb":178302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1058,"num_bytes_recovered":4329491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81514,"ondisk_log_size":81514,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1044952,"kb_used_data":657000,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282070568,"statfs":{"total":289910292480,"available":288840261632,"internally_reserved":0,"allocated":672768000,"data_stored":671949931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":397200385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,7,9,8,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28880896,"num_objects":7051,"num_object_clones":0,"num_object_copies":14102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-418,"num_objects_unfound":0,"num_objects_dirty":7051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":1712128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004603"},"pg_stats":[{"pgid":"2.f","version":"43'5192","reported_seq":5265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.692906+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:53.692906+0000","last_peered":"2024-09-11T05:14:53.692906+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:53.692906+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5192,"log_dups_size":0,"ondisk_log_size":5192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21266432,"num_objects":5192,"num_object_clones":0,"num_object_copies":10384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5192,"num_write_kb":20768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1517},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5200","reported_seq":5297,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:55.593428+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:55.593428+0000","last_peered":"2024-09-11T05:14:55.593428+0000","last_clean":"2024-09-11T05:14:55.593428+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:55.593428+0000","last_undegraded":"2024-09-11T05:14:55.593428+0000","last_fullsized":"2024-09-11T05:14:55.593428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5200,"log_dups_size":0,"ondisk_log_size":5200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5074","reported_seq":5192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:55.671371+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:55.671371+0000","last_peered":"2024-09-11T05:14:55.671371+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:55.671371+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5074,"log_dups_size":0,"ondisk_log_size":5074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3477,"num_write_kb":13908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1576}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5234","reported_seq":5300,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:55.545290+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:55.545290+0000","last_peered":"2024-09-11T05:14:55.545290+0000","last_clean":"2024-09-11T05:14:55.545290+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:55.545290+0000","last_undegraded":"2024-09-11T05:14:55.545290+0000","last_fullsized":"2024-09-11T05:14:55.545290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5234,"log_dups_size":0,"ondisk_log_size":5234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5134","reported_seq":5258,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:53.743347+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:53.743347+0000","last_peered":"2024-09-11T05:14:53.743347+0000","last_clean":"2024-09-11T05:14:53.743347+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:53.743347+0000","last_undegraded":"2024-09-11T05:14:53.743347+0000","last_fullsized":"2024-09-11T05:14:53.743347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5134,"log_dups_size":0,"ondisk_log_size":5134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21000192,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1968,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5110","reported_seq":5254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.735211+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:53.735211+0000","last_peered":"2024-09-11T05:14:53.735211+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:53.735211+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5110,"log_dups_size":0,"ondisk_log_size":5110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20910080,"num_objects":5105,"num_object_clones":0,"num_object_copies":10210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1654,"num_write_kb":6616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1505},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5074","reported_seq":5200,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.768022+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:53.768022+0000","last_peered":"2024-09-11T05:14:53.768022+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:53.768022+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5074,"log_dups_size":0,"ondisk_log_size":5074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20758528,"num_objects":5068,"num_object_clones":0,"num_object_copies":10136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2239,"num_write_kb":8956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1492},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5030","reported_seq":5149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:53.776031+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:53.776031+0000","last_peered":"2024-09-11T05:14:53.776031+0000","last_clean":"2024-09-11T05:14:53.776031+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:53.776031+0000","last_undegraded":"2024-09-11T05:14:53.776031+0000","last_fullsized":"2024-09-11T05:14:53.776031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5030,"log_dups_size":0,"ondisk_log_size":5030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20594688,"num_objects":5028,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2333,"num_write_kb":9332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5125","reported_seq":5251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:55.635334+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:55.635334+0000","last_peered":"2024-09-11T05:14:55.635334+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:55.635334+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5125,"log_dups_size":0,"ondisk_log_size":5125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3193,"num_write_kb":12772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4940","reported_seq":5097,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.733517+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:53.733517+0000","last_peered":"2024-09-11T05:14:53.733517+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:53.733517+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":4940,"log_dups_size":0,"ondisk_log_size":4940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20201472,"num_objects":4932,"num_object_clones":0,"num_object_copies":9864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1693,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1490},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.904511+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.904511+0000","last_peered":"2024-09-11T05:14:13.904511+0000","last_clean":"2024-09-11T05:14:13.904511+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.904511+0000","last_undegraded":"2024-09-11T05:14:13.904511+0000","last_fullsized":"2024-09-11T05:14:13.904511+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5144","reported_seq":5256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:55.619804+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:55.619804+0000","last_peered":"2024-09-11T05:14:55.619804+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:55.619804+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5144,"log_dups_size":0,"ondisk_log_size":5144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3173,"num_write_kb":12692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5078","reported_seq":7303,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:55.662518+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:55.662518+0000","last_peered":"2024-09-11T05:14:55.662518+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:55.662518+0000","last_undegraded":"2024-09-11T05:14:55.662518+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2422,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2981,"num_write_kb":11924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":4325376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2422},{"shards":"2","objects":2650}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5196","reported_seq":5322,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.750146+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:53.750146+0000","last_peered":"2024-09-11T05:14:53.750146+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:53.750146+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5196,"log_dups_size":0,"ondisk_log_size":5196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":8916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4972","reported_seq":5097,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.760311+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:53.760311+0000","last_peered":"2024-09-11T05:14:53.760311+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:53.760311+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20332544,"num_objects":4964,"num_object_clones":0,"num_object_copies":9928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1469},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4983","reported_seq":5113,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.773972+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:53.773972+0000","last_peered":"2024-09-11T05:14:53.773972+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:53.773972+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":4983,"log_dups_size":0,"ondisk_log_size":4983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1741,"num_write_kb":6964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1435},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5025","reported_seq":5163,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:53.758859+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:53.758859+0000","last_peered":"2024-09-11T05:14:53.758859+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:53.758859+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5025,"log_dups_size":0,"ondisk_log_size":5025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20533865,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2032,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1499},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":333533801,"num_objects":81430,"num_object_clones":0,"num_object_copies":162860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2422,"num_objects_unfound":0,"num_objects_dirty":81430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44575,"num_write_kb":178300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1057,"num_bytes_recovered":4329472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":674537472,"data_stored":674450497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81511,"ondisk_log_size":81511,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417824,"kb_used_data":277840,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93954016,"statfs":{"total":96636764160,"available":96208912384,"internally_reserved":0,"allocated":284508160,"data_stored":284263701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166160,"kb_used_data":80320,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94205680,"statfs":{"total":96636764160,"available":96466616320,"internally_reserved":0,"allocated":82247680,"data_stored":81922606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215133,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460968,"kb_used_data":298840,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93910872,"statfs":{"total":96636764160,"available":96164732928,"internally_reserved":0,"allocated":306012160,"data_stored":305763624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":165998251},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306851840,"data_stored":306848361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":82599936,"data_stored":82519919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285085696,"data_stored":285082217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:14:59.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:00.084 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:00.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 87 19 84716 84697 3.80136 2.16016 0.0373019 0.0205433 2024-09-11T05:15:00.142 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-11T05:14:59.385974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":341213820,"num_objects":83307,"num_object_clones":0,"num_object_copies":166614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2422,"num_objects_unfound":0,"num_objects_dirty":83307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46452,"num_write_kb":185802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1058,"num_bytes_recovered":4329491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83389,"ondisk_log_size":83389,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1063656,"kb_used_data":665976,"kb_used_omap":12,"kb_used_meta":397619,"kb_avail":282051864,"statfs":{"total":289910292480,"available":288821108736,"internally_reserved":0,"allocated":681959424,"data_stored":681141355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":407161857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,4,5,13,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7759,"num_write_kb":31036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005297"},"pg_stats":[{"pgid":"2.f","version":"43'5369","reported_seq":5437,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.757692+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:58.757692+0000","last_peered":"2024-09-11T05:14:58.757692+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:58.757692+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5369,"log_dups_size":0,"ondisk_log_size":5369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21991424,"num_objects":5369,"num_object_clones":0,"num_object_copies":10738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5369,"num_write_kb":21476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5200","reported_seq":5297,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:55.593428+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:14:55.593428+0000","last_peered":"2024-09-11T05:14:55.593428+0000","last_clean":"2024-09-11T05:14:55.593428+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:14:55.593428+0000","last_undegraded":"2024-09-11T05:14:55.593428+0000","last_fullsized":"2024-09-11T05:14:55.593428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5200,"log_dups_size":0,"ondisk_log_size":5200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5074","reported_seq":5192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:55.671371+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:14:55.671371+0000","last_peered":"2024-09-11T05:14:55.671371+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:14:55.671371+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5074,"log_dups_size":0,"ondisk_log_size":5074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3477,"num_write_kb":13908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1576}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5234","reported_seq":5300,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:55.545290+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:14:55.545290+0000","last_peered":"2024-09-11T05:14:55.545290+0000","last_clean":"2024-09-11T05:14:55.545290+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:14:55.545290+0000","last_undegraded":"2024-09-11T05:14:55.545290+0000","last_fullsized":"2024-09-11T05:14:55.545290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5234,"log_dups_size":0,"ondisk_log_size":5234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5343","reported_seq":5467,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:58.829967+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:58.829967+0000","last_peered":"2024-09-11T05:14:58.829967+0000","last_clean":"2024-09-11T05:14:58.829967+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:58.829967+0000","last_undegraded":"2024-09-11T05:14:58.829967+0000","last_fullsized":"2024-09-11T05:14:58.829967+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856256,"num_objects":5336,"num_object_clones":0,"num_object_copies":10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2177,"num_write_kb":8708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5300","reported_seq":5443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.805329+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:58.805329+0000","last_peered":"2024-09-11T05:14:58.805329+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:58.805329+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1844,"num_write_kb":7376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1695},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5274","reported_seq":5401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.775424+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:58.775424+0000","last_peered":"2024-09-11T05:14:58.775424+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:58.775424+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5274,"log_dups_size":0,"ondisk_log_size":5274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21577728,"num_objects":5268,"num_object_clones":0,"num_object_copies":10536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1692},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5208","reported_seq":5327,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:58.752421+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:58.752421+0000","last_peered":"2024-09-11T05:14:58.752421+0000","last_clean":"2024-09-11T05:14:58.752421+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:58.752421+0000","last_undegraded":"2024-09-11T05:14:58.752421+0000","last_fullsized":"2024-09-11T05:14:58.752421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2511,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5125","reported_seq":5251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:55.635334+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:14:55.635334+0000","last_peered":"2024-09-11T05:14:55.635334+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:14:55.635334+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5125,"log_dups_size":0,"ondisk_log_size":5125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3193,"num_write_kb":12772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5133","reported_seq":5293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.706914+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:58.706914+0000","last_peered":"2024-09-11T05:14:58.706914+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:58.706914+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20992000,"num_objects":5125,"num_object_clones":0,"num_object_copies":10250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1886,"num_write_kb":7544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5144","reported_seq":5256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:55.619804+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:14:55.619804+0000","last_peered":"2024-09-11T05:14:55.619804+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:14:55.619804+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5144,"log_dups_size":0,"ondisk_log_size":5144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3173,"num_write_kb":12692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5078","reported_seq":7303,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:14:55.662518+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:14:55.662518+0000","last_peered":"2024-09-11T05:14:55.662518+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:14:55.662518+0000","last_undegraded":"2024-09-11T05:14:55.662518+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2422,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2981,"num_write_kb":11924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1056,"num_bytes_recovered":4325376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2422},{"shards":"2","objects":2650}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5376","reported_seq":5502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.785969+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:58.785969+0000","last_peered":"2024-09-11T05:14:58.785969+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:58.785969+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22011904,"num_objects":5374,"num_object_clones":0,"num_object_copies":10748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2409,"num_write_kb":9636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1751},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5166","reported_seq":5294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.819484+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:58.819484+0000","last_peered":"2024-09-11T05:14:58.819484+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:58.819484+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21127168,"num_objects":5158,"num_object_clones":0,"num_object_copies":10316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2032,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1663},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5190","reported_seq":5320,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.760304+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:58.760304+0000","last_peered":"2024-09-11T05:14:58.760304+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:58.760304+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5190,"log_dups_size":0,"ondisk_log_size":5190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21241856,"num_objects":5186,"num_object_clones":0,"num_object_copies":10372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1642},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5172","reported_seq":5310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.816027+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:58.816027+0000","last_peered":"2024-09-11T05:14:58.816027+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:58.816027+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5172,"log_dups_size":0,"ondisk_log_size":5172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135977,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":8716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1646},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":341213801,"num_objects":83305,"num_object_clones":0,"num_object_copies":166610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2422,"num_objects_unfound":0,"num_objects_dirty":83305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46450,"num_write_kb":185800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1057,"num_bytes_recovered":4329472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":683769856,"data_stored":683682881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83386,"ondisk_log_size":83386,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417824,"kb_used_data":277840,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93954016,"statfs":{"total":96636764160,"available":96208912384,"internally_reserved":0,"allocated":284508160,"data_stored":284263701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166160,"kb_used_data":80320,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94205680,"statfs":{"total":96636764160,"available":96466616320,"internally_reserved":0,"allocated":82247680,"data_stored":81922606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":87879339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215134,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479672,"kb_used_data":307816,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93892168,"statfs":{"total":96636764160,"available":96145580032,"internally_reserved":0,"allocated":315203584,"data_stored":314955048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316084224,"data_stored":316080745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":82599936,"data_stored":82519919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285085696,"data_stored":285082217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5369", 'reported_seq': 5437, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.757692+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:14:58.757692+0000', 'last_peered': '2024-09-11T05:14:58.757692+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:14:58.757692+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 5369, 'log_dups_size': 0, 'ondisk_log_size': 5369, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21991424, 'num_objects': 5369, 'num_object_clones': 0, 'num_object_copies': 10738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5369, 'num_write_kb': 21476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1694}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5074", 'reported_seq': 5192, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:55.671371+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:14:55.671371+0000', 'last_peered': '2024-09-11T05:14:55.671371+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:14:55.671371+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 5074, 'log_dups_size': 0, 'ondisk_log_size': 5074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20766720, 'num_objects': 5070, 'num_object_clones': 0, 'num_object_copies': 10140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5070, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3477, 'num_write_kb': 13908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 1576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5300", 'reported_seq': 5443, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.805329+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:14:58.805329+0000', 'last_peered': '2024-09-11T05:14:58.805329+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:14:58.805329+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 5300, 'log_dups_size': 0, 'ondisk_log_size': 5300, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21688320, 'num_objects': 5295, 'num_object_clones': 0, 'num_object_copies': 10590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1844, 'num_write_kb': 7376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1695}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5274", 'reported_seq': 5401, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.775424+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:14:58.775424+0000', 'last_peered': '2024-09-11T05:14:58.775424+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:14:58.775424+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 5274, 'log_dups_size': 0, 'ondisk_log_size': 5274, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21577728, 'num_objects': 5268, 'num_object_clones': 0, 'num_object_copies': 10536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2439, 'num_write_kb': 9756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1692}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5125", 'reported_seq': 5251, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:55.635334+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:14:55.635334+0000', 'last_peered': '2024-09-11T05:14:55.635334+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:14:55.635334+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 5125, 'log_dups_size': 0, 'ondisk_log_size': 5125, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20963328, 'num_objects': 5118, 'num_object_clones': 0, 'num_object_copies': 10236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3193, 'num_write_kb': 12772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5133", 'reported_seq': 5293, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.706914+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:14:58.706914+0000', 'last_peered': '2024-09-11T05:14:58.706914+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:14:58.706914+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 5133, 'log_dups_size': 0, 'ondisk_log_size': 5133, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20992000, 'num_objects': 5125, 'num_object_clones': 0, 'num_object_copies': 10250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1886, 'num_write_kb': 7544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1683}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5144", 'reported_seq': 5256, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:55.619804+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:14:55.619804+0000', 'last_peered': '2024-09-11T05:14:55.619804+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:14:55.619804+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 5144, 'log_dups_size': 0, 'ondisk_log_size': 5144, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21049344, 'num_objects': 5139, 'num_object_clones': 0, 'num_object_copies': 10278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3173, 'num_write_kb': 12692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 1617}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5078", 'reported_seq': 7303, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:14:55.662518+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:14:55.662518+0000', 'last_peered': '2024-09-11T05:14:55.662518+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:14:55.662518+0000', 'last_undegraded': '2024-09-11T05:14:55.662518+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 5078, 'log_dups_size': 0, 'ondisk_log_size': 5078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20774912, 'num_objects': 5072, 'num_object_clones': 0, 'num_object_copies': 10144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2422, 'num_objects_unfound': 0, 'num_objects_dirty': 5072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2981, 'num_write_kb': 11924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1056, 'num_bytes_recovered': 4325376, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2422}, {'shards': '2', 'objects': 2650}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5376", 'reported_seq': 5502, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.785969+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:14:58.785969+0000', 'last_peered': '2024-09-11T05:14:58.785969+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:14:58.785969+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 5376, 'log_dups_size': 0, 'ondisk_log_size': 5376, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22011904, 'num_objects': 5374, 'num_object_clones': 0, 'num_object_copies': 10748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5374, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2409, 'num_write_kb': 9636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1751}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:00.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5166", 'reported_seq': 5294, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.819484+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:14:58.819484+0000', 'last_peered': '2024-09-11T05:14:58.819484+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:14:58.819484+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 5166, 'log_dups_size': 0, 'ondisk_log_size': 5166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21127168, 'num_objects': 5158, 'num_object_clones': 0, 'num_object_copies': 10316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2032, 'num_write_kb': 8128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1663}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:00.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5190", 'reported_seq': 5320, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.760304+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:14:58.760304+0000', 'last_peered': '2024-09-11T05:14:58.760304+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:14:58.760304+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 5190, 'log_dups_size': 0, 'ondisk_log_size': 5190, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21241856, 'num_objects': 5186, 'num_object_clones': 0, 'num_object_copies': 10372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1948, 'num_write_kb': 7792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1642}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:00.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5172", 'reported_seq': 5310, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:14:58.816027+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:14:58.816027+0000', 'last_peered': '2024-09-11T05:14:58.816027+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:14:58.816027+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 5172, 'log_dups_size': 0, 'ondisk_log_size': 5172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21135977, 'num_objects': 5161, 'num_object_clones': 0, 'num_object_copies': 10322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2179, 'num_write_kb': 8716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1646}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:00.145 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:01.074 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:01.075 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/166614 objects degraded (23.426%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":178,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":83307,"data_bytes":341213820,"bytes_used":1089183744,"bytes_avail":288821108736,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":166614,"degraded_ratio":0.23426002616826916,"misplaced_objects":2422,"misplaced_total":166614,"misplaced_ratio":0.014536593563566087,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96895,"recovering_keys_per_sec":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"write_bytes_sec":2647236,"read_op_per_sec":0,"write_op_per_sec":646},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:15:01.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 88 20 85263 85243 3.7824 2.13281 0.0100635 0.0206433 2024-09-11T05:15:01.124 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:01.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:01.978 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:01.980 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:02.041 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-11T05:15:01.386426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345711228,"num_objects":84405,"num_object_clones":0,"num_object_copies":168810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":84405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47550,"num_write_kb":190194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":4952083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84487,"ondisk_log_size":84487,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1109372,"kb_used_data":681160,"kb_used_omap":12,"kb_used_meta":428147,"kb_avail":282006148,"statfs":{"total":289910292480,"available":288774295552,"internally_reserved":0,"allocated":697507840,"data_stored":696689771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":438422529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,6,3,14,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28786688,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-436,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7028,"num_write_kb":28112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":436,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005264"},"pg_stats":[{"pgid":"2.f","version":"43'5369","reported_seq":5437,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.757692+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:14:58.757692+0000","last_peered":"2024-09-11T05:14:58.757692+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:14:58.757692+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5369,"log_dups_size":0,"ondisk_log_size":5369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21991424,"num_objects":5369,"num_object_clones":0,"num_object_copies":10738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5369,"num_write_kb":21476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5391","reported_seq":5487,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:00.678445+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:00.678445+0000","last_peered":"2024-09-11T05:15:00.678445+0000","last_clean":"2024-09-11T05:15:00.678445+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:00.678445+0000","last_undegraded":"2024-09-11T05:15:00.678445+0000","last_fullsized":"2024-09-11T05:15:00.678445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5391,"log_dups_size":0,"ondisk_log_size":5391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3789,"num_write_kb":15156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5246","reported_seq":5366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.603872+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:00.603872+0000","last_peered":"2024-09-11T05:15:00.603872+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:00.603872+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21471232,"num_objects":5242,"num_object_clones":0,"num_object_copies":10484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3649,"num_write_kb":14596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5406","reported_seq":5472,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:00.612211+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:00.612211+0000","last_peered":"2024-09-11T05:15:00.612211+0000","last_clean":"2024-09-11T05:15:00.612211+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:00.612211+0000","last_undegraded":"2024-09-11T05:15:00.612211+0000","last_fullsized":"2024-09-11T05:15:00.612211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5406,"log_dups_size":0,"ondisk_log_size":5406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22142976,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5406,"num_write_kb":21624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5343","reported_seq":5467,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:58.829967+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:14:58.829967+0000","last_peered":"2024-09-11T05:14:58.829967+0000","last_clean":"2024-09-11T05:14:58.829967+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:14:58.829967+0000","last_undegraded":"2024-09-11T05:14:58.829967+0000","last_fullsized":"2024-09-11T05:14:58.829967+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856256,"num_objects":5336,"num_object_clones":0,"num_object_copies":10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2177,"num_write_kb":8708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5300","reported_seq":5443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.805329+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:14:58.805329+0000","last_peered":"2024-09-11T05:14:58.805329+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:14:58.805329+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1844,"num_write_kb":7376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1695},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5274","reported_seq":5401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.775424+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:14:58.775424+0000","last_peered":"2024-09-11T05:14:58.775424+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:14:58.775424+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5274,"log_dups_size":0,"ondisk_log_size":5274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21577728,"num_objects":5268,"num_object_clones":0,"num_object_copies":10536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1692},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5208","reported_seq":5327,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:58.752421+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:14:58.752421+0000","last_peered":"2024-09-11T05:14:58.752421+0000","last_clean":"2024-09-11T05:14:58.752421+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:14:58.752421+0000","last_undegraded":"2024-09-11T05:14:58.752421+0000","last_fullsized":"2024-09-11T05:14:58.752421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2511,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5317","reported_seq":5443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.616843+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:00.616843+0000","last_peered":"2024-09-11T05:15:00.616843+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:00.616843+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5317,"log_dups_size":0,"ondisk_log_size":5317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5133","reported_seq":5293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.706914+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:14:58.706914+0000","last_peered":"2024-09-11T05:14:58.706914+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:14:58.706914+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20992000,"num_objects":5125,"num_object_clones":0,"num_object_copies":10250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1886,"num_write_kb":7544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5327","reported_seq":5439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.545024+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:00.545024+0000","last_peered":"2024-09-11T05:15:00.545024+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:00.545024+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21798912,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3356,"num_write_kb":13424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5266","reported_seq":7795,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:00.675088+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:00.675088+0000","last_peered":"2024-09-11T05:15:00.675088+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:00.675088+0000","last_undegraded":"2024-09-11T05:15:00.675088+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5266,"log_dups_size":0,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3169,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":4947968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2270},{"shards":"2","objects":2990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5376","reported_seq":5502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.785969+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:14:58.785969+0000","last_peered":"2024-09-11T05:14:58.785969+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:14:58.785969+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22011904,"num_objects":5374,"num_object_clones":0,"num_object_copies":10748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2409,"num_write_kb":9636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1751},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5166","reported_seq":5294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.819484+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:14:58.819484+0000","last_peered":"2024-09-11T05:14:58.819484+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:14:58.819484+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21127168,"num_objects":5158,"num_object_clones":0,"num_object_copies":10316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2032,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1663},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5190","reported_seq":5320,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.760304+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:14:58.760304+0000","last_peered":"2024-09-11T05:14:58.760304+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:14:58.760304+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5190,"log_dups_size":0,"ondisk_log_size":5190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21241856,"num_objects":5186,"num_object_clones":0,"num_object_copies":10372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1642},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5172","reported_seq":5310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:14:58.816027+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:14:58.816027+0000","last_peered":"2024-09-11T05:14:58.816027+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:14:58.816027+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5172,"log_dups_size":0,"ondisk_log_size":5172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135977,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":8716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1646},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345711209,"num_objects":84403,"num_object_clones":0,"num_object_copies":168806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":84403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47548,"num_write_kb":190192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1209,"num_bytes_recovered":4952064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":699285504,"data_stored":699198529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84484,"ondisk_log_size":84484,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427756,"kb_used_data":283612,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93944084,"statfs":{"total":96636764160,"available":96198742016,"internally_reserved":0,"allocated":290418688,"data_stored":290174229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147582635},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201944,"kb_used_data":89732,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94169896,"statfs":{"total":96636764160,"available":96429973504,"internally_reserved":0,"allocated":91885568,"data_stored":91560494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":114880171},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,1,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215134,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479672,"kb_used_data":307816,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93892168,"statfs":{"total":96636764160,"available":96145580032,"internally_reserved":0,"allocated":315203584,"data_stored":314955048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316084224,"data_stored":316080745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92209152,"data_stored":92129135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290992128,"data_stored":290988649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:02.098 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 89 20 85804 85784 3.76365 2.11328 0.0481379 0.0207483 2024-09-11T05:15:03.099 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 90 20 86401 86381 3.74774 2.33203 0.00772918 0.0208375 2024-09-11T05:15:04.103 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 91 20 86978 86958 3.73116 2.25391 0.0421387 0.0209305 2024-09-11T05:15:05.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:05.103 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 92 20 87567 87547 3.71561 2.30078 0.0568369 0.021018 2024-09-11T05:15:05.917 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:05.918 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:05.988 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-11T05:15:05.387358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352973436,"num_objects":86178,"num_object_clones":0,"num_object_copies":172356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":86178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49323,"num_write_kb":197286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":4952083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86260,"ondisk_log_size":86260,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1121844,"kb_used_data":689472,"kb_used_omap":12,"kb_used_meta":432307,"kb_avail":281993676,"statfs":{"total":289910292480,"available":288761524224,"internally_reserved":0,"allocated":706019328,"data_stored":705201259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":442682369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,5,8,5,5,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-288,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":30660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003581"},"pg_stats":[{"pgid":"2.f","version":"43'5525","reported_seq":5598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.807624+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:03.807624+0000","last_peered":"2024-09-11T05:15:03.807624+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:03.807624+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5525,"log_dups_size":0,"ondisk_log_size":5525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22630400,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5525,"num_write_kb":22100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1850},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5391","reported_seq":5487,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:00.678445+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:00.678445+0000","last_peered":"2024-09-11T05:15:00.678445+0000","last_clean":"2024-09-11T05:15:00.678445+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:00.678445+0000","last_undegraded":"2024-09-11T05:15:00.678445+0000","last_fullsized":"2024-09-11T05:15:00.678445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5391,"log_dups_size":0,"ondisk_log_size":5391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3789,"num_write_kb":15156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5246","reported_seq":5366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.603872+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:00.603872+0000","last_peered":"2024-09-11T05:15:00.603872+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:00.603872+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21471232,"num_objects":5242,"num_object_clones":0,"num_object_copies":10484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3649,"num_write_kb":14596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5406","reported_seq":5472,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:00.612211+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:00.612211+0000","last_peered":"2024-09-11T05:15:00.612211+0000","last_clean":"2024-09-11T05:15:00.612211+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:00.612211+0000","last_undegraded":"2024-09-11T05:15:00.612211+0000","last_fullsized":"2024-09-11T05:15:00.612211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5406,"log_dups_size":0,"ondisk_log_size":5406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22142976,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5406,"num_write_kb":21624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5518","reported_seq":5642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:03.847299+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:03.847299+0000","last_peered":"2024-09-11T05:15:03.847299+0000","last_clean":"2024-09-11T05:15:03.847299+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:03.847299+0000","last_undegraded":"2024-09-11T05:15:03.847299+0000","last_fullsized":"2024-09-11T05:15:03.847299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5469","reported_seq":5612,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.803698+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:03.803698+0000","last_peered":"2024-09-11T05:15:03.803698+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:03.803698+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5469,"log_dups_size":0,"ondisk_log_size":5469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22380544,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2013,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1864},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5472","reported_seq":5598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.844803+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:03.844803+0000","last_peered":"2024-09-11T05:15:03.844803+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:03.844803+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5472,"log_dups_size":0,"ondisk_log_size":5472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22388736,"num_objects":5466,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2637,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1890},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5384","reported_seq":5503,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:03.789671+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:03.789671+0000","last_peered":"2024-09-11T05:15:03.789671+0000","last_clean":"2024-09-11T05:15:03.789671+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:03.789671+0000","last_undegraded":"2024-09-11T05:15:03.789671+0000","last_fullsized":"2024-09-11T05:15:03.789671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5384,"log_dups_size":0,"ondisk_log_size":5384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22044672,"num_objects":5382,"num_object_clones":0,"num_object_copies":10764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5317","reported_seq":5443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.616843+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:00.616843+0000","last_peered":"2024-09-11T05:15:00.616843+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:00.616843+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5317,"log_dups_size":0,"ondisk_log_size":5317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5327","reported_seq":5487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.793336+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:03.793336+0000","last_peered":"2024-09-11T05:15:03.793336+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:03.793336+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2080,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5327","reported_seq":5439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.545024+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:00.545024+0000","last_peered":"2024-09-11T05:15:00.545024+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:00.545024+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21798912,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3356,"num_write_kb":13424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5266","reported_seq":7795,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:00.675088+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:00.675088+0000","last_peered":"2024-09-11T05:15:00.675088+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:00.675088+0000","last_undegraded":"2024-09-11T05:15:00.675088+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5266,"log_dups_size":0,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3169,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":4947968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2270},{"shards":"2","objects":2990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5548","reported_seq":5674,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.854676+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:03.854676+0000","last_peered":"2024-09-11T05:15:03.854676+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:03.854676+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5548,"log_dups_size":0,"ondisk_log_size":5548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22716416,"num_objects":5546,"num_object_clones":0,"num_object_copies":11092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2581,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5348","reported_seq":5476,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.847813+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:03.847813+0000","last_peered":"2024-09-11T05:15:03.847813+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:03.847813+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2214,"num_write_kb":8856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1845},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5371","reported_seq":5500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.849429+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:03.849429+0000","last_peered":"2024-09-11T05:15:03.849429+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:03.849429+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5371,"log_dups_size":0,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983232,"num_objects":5367,"num_object_clones":0,"num_object_copies":10734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2129,"num_write_kb":8516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1823},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5342","reported_seq":5480,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.620287+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:03.620287+0000","last_peered":"2024-09-11T05:15:03.620287+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:03.620287+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5342,"log_dups_size":0,"ondisk_log_size":5342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21832297,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2349,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1816},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352973417,"num_objects":86176,"num_object_clones":0,"num_object_copies":172352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":86176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49321,"num_write_kb":197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1209,"num_bytes_recovered":4952064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":707891200,"data_stored":707804225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86257,"ondisk_log_size":86257,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427756,"kb_used_data":283612,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93944084,"statfs":{"total":96636764160,"available":96198742016,"internally_reserved":0,"allocated":290418688,"data_stored":290174229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147582635},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201944,"kb_used_data":89732,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94169896,"statfs":{"total":96636764160,"available":96429973504,"internally_reserved":0,"allocated":91885568,"data_stored":91560494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":114880171},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,1,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215135,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492144,"kb_used_data":316128,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93879696,"statfs":{"total":96636764160,"available":96132808704,"internally_reserved":0,"allocated":323715072,"data_stored":323466536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,3,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":324689920,"data_stored":324686441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92209152,"data_stored":92129135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290992128,"data_stored":290988649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:05.990 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:06.107 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 93 20 88128 88108 3.69906 2.19141 0.0146848 0.0211113 2024-09-11T05:15:06.889 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:06.889 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:06.944 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-11T05:15:05.387358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352973436,"num_objects":86178,"num_object_clones":0,"num_object_copies":172356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":86178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49323,"num_write_kb":197286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":4952083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86260,"ondisk_log_size":86260,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1121844,"kb_used_data":689472,"kb_used_omap":12,"kb_used_meta":432307,"kb_avail":281993676,"statfs":{"total":289910292480,"available":288761524224,"internally_reserved":0,"allocated":706019328,"data_stored":705201259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":442682369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,5,8,5,5,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-288,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":30660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003581"},"pg_stats":[{"pgid":"2.f","version":"43'5525","reported_seq":5598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.807624+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:03.807624+0000","last_peered":"2024-09-11T05:15:03.807624+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:03.807624+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5525,"log_dups_size":0,"ondisk_log_size":5525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22630400,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5525,"num_write_kb":22100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1850},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5391","reported_seq":5487,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:00.678445+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:00.678445+0000","last_peered":"2024-09-11T05:15:00.678445+0000","last_clean":"2024-09-11T05:15:00.678445+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:00.678445+0000","last_undegraded":"2024-09-11T05:15:00.678445+0000","last_fullsized":"2024-09-11T05:15:00.678445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5391,"log_dups_size":0,"ondisk_log_size":5391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3789,"num_write_kb":15156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5246","reported_seq":5366,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.603872+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:00.603872+0000","last_peered":"2024-09-11T05:15:00.603872+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:00.603872+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21471232,"num_objects":5242,"num_object_clones":0,"num_object_copies":10484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3649,"num_write_kb":14596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5406","reported_seq":5472,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:00.612211+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:00.612211+0000","last_peered":"2024-09-11T05:15:00.612211+0000","last_clean":"2024-09-11T05:15:00.612211+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:00.612211+0000","last_undegraded":"2024-09-11T05:15:00.612211+0000","last_fullsized":"2024-09-11T05:15:00.612211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5406,"log_dups_size":0,"ondisk_log_size":5406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22142976,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5406,"num_write_kb":21624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5518","reported_seq":5642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:03.847299+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:03.847299+0000","last_peered":"2024-09-11T05:15:03.847299+0000","last_clean":"2024-09-11T05:15:03.847299+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:03.847299+0000","last_undegraded":"2024-09-11T05:15:03.847299+0000","last_fullsized":"2024-09-11T05:15:03.847299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5469","reported_seq":5612,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.803698+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:03.803698+0000","last_peered":"2024-09-11T05:15:03.803698+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:03.803698+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5469,"log_dups_size":0,"ondisk_log_size":5469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22380544,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2013,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1864},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5472","reported_seq":5598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.844803+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:03.844803+0000","last_peered":"2024-09-11T05:15:03.844803+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:03.844803+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5472,"log_dups_size":0,"ondisk_log_size":5472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22388736,"num_objects":5466,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2637,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1890},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5384","reported_seq":5503,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:03.789671+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:03.789671+0000","last_peered":"2024-09-11T05:15:03.789671+0000","last_clean":"2024-09-11T05:15:03.789671+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:03.789671+0000","last_undegraded":"2024-09-11T05:15:03.789671+0000","last_fullsized":"2024-09-11T05:15:03.789671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5384,"log_dups_size":0,"ondisk_log_size":5384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22044672,"num_objects":5382,"num_object_clones":0,"num_object_copies":10764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5317","reported_seq":5443,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.616843+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:00.616843+0000","last_peered":"2024-09-11T05:15:00.616843+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:00.616843+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5317,"log_dups_size":0,"ondisk_log_size":5317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5327","reported_seq":5487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.793336+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:03.793336+0000","last_peered":"2024-09-11T05:15:03.793336+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:03.793336+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2080,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5327","reported_seq":5439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:00.545024+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:00.545024+0000","last_peered":"2024-09-11T05:15:00.545024+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:00.545024+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21798912,"num_objects":5322,"num_object_clones":0,"num_object_copies":10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3356,"num_write_kb":13424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":1800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5266","reported_seq":7795,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:00.675088+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:00.675088+0000","last_peered":"2024-09-11T05:15:00.675088+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:00.675088+0000","last_undegraded":"2024-09-11T05:15:00.675088+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5266,"log_dups_size":0,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3169,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1208,"num_bytes_recovered":4947968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2270},{"shards":"2","objects":2990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5548","reported_seq":5674,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.854676+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:03.854676+0000","last_peered":"2024-09-11T05:15:03.854676+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:03.854676+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5548,"log_dups_size":0,"ondisk_log_size":5548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22716416,"num_objects":5546,"num_object_clones":0,"num_object_copies":11092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2581,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5348","reported_seq":5476,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.847813+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:03.847813+0000","last_peered":"2024-09-11T05:15:03.847813+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:03.847813+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2214,"num_write_kb":8856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1845},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5371","reported_seq":5500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.849429+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:03.849429+0000","last_peered":"2024-09-11T05:15:03.849429+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:03.849429+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5371,"log_dups_size":0,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983232,"num_objects":5367,"num_object_clones":0,"num_object_copies":10734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2129,"num_write_kb":8516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1823},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5342","reported_seq":5480,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.620287+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:03.620287+0000","last_peered":"2024-09-11T05:15:03.620287+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:03.620287+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5342,"log_dups_size":0,"ondisk_log_size":5342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21832297,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2349,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1816},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352973417,"num_objects":86176,"num_object_clones":0,"num_object_copies":172352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2270,"num_objects_unfound":0,"num_objects_dirty":86176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49321,"num_write_kb":197284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1209,"num_bytes_recovered":4952064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":707891200,"data_stored":707804225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86257,"ondisk_log_size":86257,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427756,"kb_used_data":283612,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93944084,"statfs":{"total":96636764160,"available":96198742016,"internally_reserved":0,"allocated":290418688,"data_stored":290174229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147582635},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201944,"kb_used_data":89732,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94169896,"statfs":{"total":96636764160,"available":96429973504,"internally_reserved":0,"allocated":91885568,"data_stored":91560494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":114880171},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,1,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215135,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492144,"kb_used_data":316128,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93879696,"statfs":{"total":96636764160,"available":96132808704,"internally_reserved":0,"allocated":323715072,"data_stored":323466536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,3,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":324689920,"data_stored":324686441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92209152,"data_stored":92129135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290992128,"data_stored":290988649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5525", 'reported_seq': 5598, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.807624+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:03.807624+0000', 'last_peered': '2024-09-11T05:15:03.807624+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:03.807624+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 5525, 'log_dups_size': 0, 'ondisk_log_size': 5525, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22630400, 'num_objects': 5525, 'num_object_clones': 0, 'num_object_copies': 11050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5525, 'num_write_kb': 22100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1850}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5246", 'reported_seq': 5366, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:00.603872+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:00.603872+0000', 'last_peered': '2024-09-11T05:15:00.603872+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:00.603872+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 5246, 'log_dups_size': 0, 'ondisk_log_size': 5246, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21471232, 'num_objects': 5242, 'num_object_clones': 0, 'num_object_copies': 10484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5242, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3649, 'num_write_kb': 14596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 1748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5469", 'reported_seq': 5612, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.803698+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:03.803698+0000', 'last_peered': '2024-09-11T05:15:03.803698+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:03.803698+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 5469, 'log_dups_size': 0, 'ondisk_log_size': 5469, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22380544, 'num_objects': 5464, 'num_object_clones': 0, 'num_object_copies': 10928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2013, 'num_write_kb': 8052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1864}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5472", 'reported_seq': 5598, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.844803+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:03.844803+0000', 'last_peered': '2024-09-11T05:15:03.844803+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:03.844803+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 5472, 'log_dups_size': 0, 'ondisk_log_size': 5472, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22388736, 'num_objects': 5466, 'num_object_clones': 0, 'num_object_copies': 10932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5466, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2637, 'num_write_kb': 10548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1890}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5317", 'reported_seq': 5443, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:00.616843+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:00.616843+0000', 'last_peered': '2024-09-11T05:15:00.616843+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:00.616843+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 5317, 'log_dups_size': 0, 'ondisk_log_size': 5317, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21749760, 'num_objects': 5310, 'num_object_clones': 0, 'num_object_copies': 10620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3385, 'num_write_kb': 13540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 1785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5327", 'reported_seq': 5487, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.793336+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:03.793336+0000', 'last_peered': '2024-09-11T05:15:03.793336+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:03.793336+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 5327, 'log_dups_size': 0, 'ondisk_log_size': 5327, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21786624, 'num_objects': 5319, 'num_object_clones': 0, 'num_object_copies': 10638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2080, 'num_write_kb': 8320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1877}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5327", 'reported_seq': 5439, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:00.545024+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:00.545024+0000', 'last_peered': '2024-09-11T05:15:00.545024+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:00.545024+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 5327, 'log_dups_size': 0, 'ondisk_log_size': 5327, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21798912, 'num_objects': 5322, 'num_object_clones': 0, 'num_object_copies': 10644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3356, 'num_write_kb': 13424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 1800}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5266", 'reported_seq': 7795, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:00.675088+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:00.675088+0000', 'last_peered': '2024-09-11T05:15:00.675088+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:00.675088+0000', 'last_undegraded': '2024-09-11T05:15:00.675088+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 5266, 'log_dups_size': 0, 'ondisk_log_size': 5266, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21544960, 'num_objects': 5260, 'num_object_clones': 0, 'num_object_copies': 10520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2270, 'num_objects_unfound': 0, 'num_objects_dirty': 5260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3169, 'num_write_kb': 12676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1208, 'num_bytes_recovered': 4947968, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2270}, {'shards': '2', 'objects': 2990}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5548", 'reported_seq': 5674, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.854676+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:03.854676+0000', 'last_peered': '2024-09-11T05:15:03.854676+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:03.854676+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 5548, 'log_dups_size': 0, 'ondisk_log_size': 5548, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22716416, 'num_objects': 5546, 'num_object_clones': 0, 'num_object_copies': 11092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2581, 'num_write_kb': 10324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1923}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:06.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5348", 'reported_seq': 5476, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.847813+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:03.847813+0000', 'last_peered': '2024-09-11T05:15:03.847813+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:03.847813+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 5348, 'log_dups_size': 0, 'ondisk_log_size': 5348, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21872640, 'num_objects': 5340, 'num_object_clones': 0, 'num_object_copies': 10680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2214, 'num_write_kb': 8856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1845}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:06.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5371", 'reported_seq': 5500, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.849429+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:03.849429+0000', 'last_peered': '2024-09-11T05:15:03.849429+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:03.849429+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 5371, 'log_dups_size': 0, 'ondisk_log_size': 5371, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21983232, 'num_objects': 5367, 'num_object_clones': 0, 'num_object_copies': 10734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5367, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2129, 'num_write_kb': 8516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1823}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:06.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5342", 'reported_seq': 5480, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:03.620287+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:03.620287+0000', 'last_peered': '2024-09-11T05:15:03.620287+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:03.620287+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 5342, 'log_dups_size': 0, 'ondisk_log_size': 5342, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21832297, 'num_objects': 5331, 'num_object_clones': 0, 'num_object_copies': 10662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2349, 'num_write_kb': 9396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1816}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:06.948 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:07.107 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 94 20 88685 88665 3.68286 2.17578 0.0152773 0.0212055 2024-09-11T05:15:07.955 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:07.955 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/174514 objects degraded (22.366%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":185,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":87257,"data_bytes":357393020,"bytes_used":1182666752,"bytes_avail":288727625728,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":174514,"degraded_ratio":0.2236554087351158,"misplaced_objects":2108,"misplaced_total":174514,"misplaced_ratio":0.012079260116666858,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":153553,"recovering_keys_per_sec":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"write_bytes_sec":2361656,"read_op_per_sec":0,"write_op_per_sec":576},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T05:14:17.373069+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:15:08.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:08.016 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:08.107 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 95 20 89214 89194 3.66585 2.06641 0.037175 0.0213019 2024-09-11T05:15:08.743 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:08.745 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:08.797 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-11T05:15:07.387826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":357393020,"num_objects":87257,"num_object_clones":0,"num_object_copies":174514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2108,"num_objects_unfound":0,"num_objects_dirty":87257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50402,"num_write_kb":201602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1372,"num_bytes_recovered":5615635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87339,"ondisk_log_size":87339,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1154948,"kb_used_data":704532,"kb_used_omap":12,"kb_used_meta":450355,"kb_avail":281960572,"statfs":{"total":289910292480,"available":288727625728,"internally_reserved":0,"allocated":721440768,"data_stored":720622699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":461163521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,6,8,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28348416,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-450,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6921,"num_write_kb":27684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003613"},"pg_stats":[{"pgid":"2.f","version":"43'5525","reported_seq":5598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.807624+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:03.807624+0000","last_peered":"2024-09-11T05:15:03.807624+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:03.807624+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5525,"log_dups_size":0,"ondisk_log_size":5525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22630400,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5525,"num_write_kb":22100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1850},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5554","reported_seq":5651,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:05.688813+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:05.688813+0000","last_peered":"2024-09-11T05:15:05.688813+0000","last_clean":"2024-09-11T05:15:05.688813+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:05.688813+0000","last_undegraded":"2024-09-11T05:15:05.688813+0000","last_fullsized":"2024-09-11T05:15:05.688813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5554,"log_dups_size":0,"ondisk_log_size":5554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22724608,"num_objects":5548,"num_object_clones":0,"num_object_copies":11096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3952,"num_write_kb":15808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5436","reported_seq":5554,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:05.639420+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:05.639420+0000","last_peered":"2024-09-11T05:15:05.639420+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:05.639420+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5436,"log_dups_size":0,"ondisk_log_size":5436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22249472,"num_objects":5432,"num_object_clones":0,"num_object_copies":10864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":15356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":1938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5576","reported_seq":5642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:05.663038+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:05.663038+0000","last_peered":"2024-09-11T05:15:05.663038+0000","last_clean":"2024-09-11T05:15:05.663038+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:05.663038+0000","last_undegraded":"2024-09-11T05:15:05.663038+0000","last_fullsized":"2024-09-11T05:15:05.663038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5576,"log_dups_size":0,"ondisk_log_size":5576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22839296,"num_objects":5576,"num_object_clones":0,"num_object_copies":11152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5518","reported_seq":5642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:03.847299+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:03.847299+0000","last_peered":"2024-09-11T05:15:03.847299+0000","last_clean":"2024-09-11T05:15:03.847299+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:03.847299+0000","last_undegraded":"2024-09-11T05:15:03.847299+0000","last_fullsized":"2024-09-11T05:15:03.847299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5469","reported_seq":5612,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.803698+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:03.803698+0000","last_peered":"2024-09-11T05:15:03.803698+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:03.803698+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5469,"log_dups_size":0,"ondisk_log_size":5469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22380544,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2013,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1864},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5472","reported_seq":5598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.844803+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:03.844803+0000","last_peered":"2024-09-11T05:15:03.844803+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:03.844803+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5472,"log_dups_size":0,"ondisk_log_size":5472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22388736,"num_objects":5466,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2637,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1890},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5384","reported_seq":5503,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:03.789671+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:03.789671+0000","last_peered":"2024-09-11T05:15:03.789671+0000","last_clean":"2024-09-11T05:15:03.789671+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:03.789671+0000","last_undegraded":"2024-09-11T05:15:03.789671+0000","last_fullsized":"2024-09-11T05:15:03.789671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5384,"log_dups_size":0,"ondisk_log_size":5384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22044672,"num_objects":5382,"num_object_clones":0,"num_object_copies":10764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5504","reported_seq":5630,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:05.708797+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:05.708797+0000","last_peered":"2024-09-11T05:15:05.708797+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:05.708797+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22515712,"num_objects":5497,"num_object_clones":0,"num_object_copies":10994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3572,"num_write_kb":14288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":1972}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5327","reported_seq":5487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.793336+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:03.793336+0000","last_peered":"2024-09-11T05:15:03.793336+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:03.793336+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2080,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5529","reported_seq":5641,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:05.663305+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:05.663305+0000","last_peered":"2024-09-11T05:15:05.663305+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:05.663305+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3558,"num_write_kb":14232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2002}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5433","reported_seq":8285,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:05.641703+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:05.641703+0000","last_peered":"2024-09-11T05:15:05.641703+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:05.641703+0000","last_undegraded":"2024-09-11T05:15:05.641703+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22228992,"num_objects":5427,"num_object_clones":0,"num_object_copies":10854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2108,"num_objects_unfound":0,"num_objects_dirty":5427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3336,"num_write_kb":13344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1370,"num_bytes_recovered":5611520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2108},{"shards":"2","objects":3319}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5548","reported_seq":5674,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.854676+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:03.854676+0000","last_peered":"2024-09-11T05:15:03.854676+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:03.854676+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5548,"log_dups_size":0,"ondisk_log_size":5548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22716416,"num_objects":5546,"num_object_clones":0,"num_object_copies":11092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2581,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5348","reported_seq":5476,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.847813+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:03.847813+0000","last_peered":"2024-09-11T05:15:03.847813+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:03.847813+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2214,"num_write_kb":8856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1845},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5371","reported_seq":5500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.849429+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:03.849429+0000","last_peered":"2024-09-11T05:15:03.849429+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:03.849429+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5371,"log_dups_size":0,"ondisk_log_size":5371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983232,"num_objects":5367,"num_object_clones":0,"num_object_copies":10734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2129,"num_write_kb":8516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1823},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5342","reported_seq":5480,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:03.620287+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:03.620287+0000","last_peered":"2024-09-11T05:15:03.620287+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:03.620287+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5342,"log_dups_size":0,"ondisk_log_size":5342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21832297,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2349,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1816},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":357393001,"num_objects":87255,"num_object_clones":0,"num_object_copies":174510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":2108,"num_objects_unfound":0,"num_objects_dirty":87255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50400,"num_write_kb":201600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1371,"num_bytes_recovered":5615616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":723439616,"data_stored":723352641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87336,"ondisk_log_size":87336,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459916,"kb_used_data":289404,"kb_used_omap":4,"kb_used_meta":170491,"kb_avail":93911924,"statfs":{"total":96636764160,"available":96165810176,"internally_reserved":0,"allocated":296349696,"data_stored":296105237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":174583467},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202888,"kb_used_data":99000,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94168952,"statfs":{"total":96636764160,"available":96429006848,"internally_reserved":0,"allocated":101376000,"data_stored":101050926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106360491},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215135,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492144,"kb_used_data":316128,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93879696,"statfs":{"total":96636764160,"available":96132808704,"internally_reserved":0,"allocated":323715072,"data_stored":323466536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,3,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":324689920,"data_stored":324686441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":101810176,"data_stored":101730159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296939520,"data_stored":296936041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:09.107 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 96 20 89761 89741 3.64992 2.13672 0.0246893 0.0213941 2024-09-11T05:15:10.108 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 97 20 90324 90304 3.63497 2.19922 0.017996 0.0214839 2024-09-11T05:15:11.108 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 98 20 90917 90897 3.62152 2.31641 0.0310435 0.0215645 2024-09-11T05:15:11.799 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:12.108 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 99 20 91460 91440 3.60637 2.12109 0.0234567 0.0216523 2024-09-11T05:15:12.595 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:12.680 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-11T05:15:11.394808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":368706172,"num_objects":90019,"num_object_clones":0,"num_object_copies":180038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":90019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53164,"num_write_kb":212650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6275091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90101,"ondisk_log_size":90101,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1199712,"kb_used_data":727084,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281915808,"statfs":{"total":289910292480,"available":288681787392,"internally_reserved":0,"allocated":744534016,"data_stored":743715947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":483904513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,8,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27492352,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-475,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6712,"num_write_kb":26848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":1945600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008833"},"pg_stats":[{"pgid":"2.f","version":"43'5691","reported_seq":5765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.895802+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:08.895802+0000","last_peered":"2024-09-11T05:15:08.895802+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:08.895802+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5691,"log_dups_size":0,"ondisk_log_size":5691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5691,"num_write_kb":22764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2016},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5712","reported_seq":5809,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:10.649322+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:10.649322+0000","last_peered":"2024-09-11T05:15:10.649322+0000","last_clean":"2024-09-11T05:15:10.649322+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:10.649322+0000","last_undegraded":"2024-09-11T05:15:10.649322+0000","last_fullsized":"2024-09-11T05:15:10.649322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5712,"log_dups_size":0,"ondisk_log_size":5712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5618","reported_seq":5738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.699400+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:10.699400+0000","last_peered":"2024-09-11T05:15:10.699400+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:10.699400+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4021,"num_write_kb":16084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5752","reported_seq":5818,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:10.634469+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:10.634469+0000","last_peered":"2024-09-11T05:15:10.634469+0000","last_clean":"2024-09-11T05:15:10.634469+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:10.634469+0000","last_undegraded":"2024-09-11T05:15:10.634469+0000","last_fullsized":"2024-09-11T05:15:10.634469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5752,"log_dups_size":0,"ondisk_log_size":5752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23560192,"num_objects":5752,"num_object_clones":0,"num_object_copies":11504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5752,"num_write_kb":23008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5704","reported_seq":5829,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:08.902338+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:08.902338+0000","last_peered":"2024-09-11T05:15:08.902338+0000","last_clean":"2024-09-11T05:15:08.902338+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:08.902338+0000","last_undegraded":"2024-09-11T05:15:08.902338+0000","last_fullsized":"2024-09-11T05:15:08.902338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2538,"num_write_kb":10152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5618","reported_seq":5762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.873280+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:08.873280+0000","last_peered":"2024-09-11T05:15:08.873280+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:08.873280+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990848,"num_objects":5613,"num_object_clones":0,"num_object_copies":11226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2013},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5676","reported_seq":5800,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.870421+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:08.870421+0000","last_peered":"2024-09-11T05:15:08.870421+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:08.870421+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5676,"log_dups_size":0,"ondisk_log_size":5676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2841,"num_write_kb":11364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2094},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5554","reported_seq":5673,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:08.760425+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:08.760425+0000","last_peered":"2024-09-11T05:15:08.760425+0000","last_clean":"2024-09-11T05:15:08.760425+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:08.760425+0000","last_undegraded":"2024-09-11T05:15:08.760425+0000","last_fullsized":"2024-09-11T05:15:08.760425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5554,"log_dups_size":0,"ondisk_log_size":5554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2857,"num_write_kb":11428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5681","reported_seq":5807,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.709867+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:10.709867+0000","last_peered":"2024-09-11T05:15:10.709867+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:10.709867+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5681,"log_dups_size":0,"ondisk_log_size":5681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23240704,"num_objects":5674,"num_object_clones":0,"num_object_copies":11348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3749,"num_write_kb":14996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5507","reported_seq":5667,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.889584+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:08.889584+0000","last_peered":"2024-09-11T05:15:08.889584+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:08.889584+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22523904,"num_objects":5499,"num_object_clones":0,"num_object_copies":10998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2260,"num_write_kb":9040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2057},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5702","reported_seq":5814,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.607950+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:10.607950+0000","last_peered":"2024-09-11T05:15:10.607950+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:10.607950+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3731,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2175}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5594","reported_seq":8768,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:10.710944+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:10.710944+0000","last_peered":"2024-09-11T05:15:10.710944+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:10.710944+0000","last_undegraded":"2024-09-11T05:15:10.710944+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3497,"num_write_kb":13988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1531,"num_bytes_recovered":6270976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1947},{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5738","reported_seq":5864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.863125+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:08.863125+0000","last_peered":"2024-09-11T05:15:08.863125+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:08.863125+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23494656,"num_objects":5736,"num_object_clones":0,"num_object_copies":11472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2771,"num_write_kb":11084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2113},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5502","reported_seq":5630,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.847801+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:08.847801+0000","last_peered":"2024-09-11T05:15:08.847801+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:08.847801+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22503424,"num_objects":5494,"num_object_clones":0,"num_object_copies":10988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2368,"num_write_kb":9472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1999},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5532","reported_seq":5662,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.887665+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:08.887665+0000","last_peered":"2024-09-11T05:15:08.887665+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:08.887665+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1984},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5517","reported_seq":5656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.884554+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:08.884554+0000","last_peered":"2024-09-11T05:15:08.884554+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:08.884554+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22549097,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2524,"num_write_kb":10096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1991},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":368706153,"num_objects":90017,"num_object_clones":0,"num_object_copies":180034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":90017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53162,"num_write_kb":212648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":6275072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":746872832,"data_stored":746785857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90098,"ondisk_log_size":90098,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453128,"kb_used_data":295096,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93918712,"statfs":{"total":96636764160,"available":96172761088,"internally_reserved":0,"allocated":302178304,"data_stored":301933845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":238048,"kb_used_data":107788,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94133792,"statfs":{"total":96636764160,"available":96393003008,"internally_reserved":0,"allocated":110374912,"data_stored":110049838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215136,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508536,"kb_used_data":324200,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93863304,"statfs":{"total":96636764160,"available":96116023296,"internally_reserved":0,"allocated":331980800,"data_stored":331732264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":333193216,"data_stored":333189737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":111042560,"data_stored":110962543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302637056,"data_stored":302633577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:12.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:13.108 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:15:13.109103+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0217446 2024-09-11T05:15:13.108 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:15:13.108 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 100 20 91998 91978 3.59132 2.10156 0.0722833 0.0217446 2024-09-11T05:15:13.590 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:13.591 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:13.656 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-11T05:15:13.395286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":368706172,"num_objects":90019,"num_object_clones":0,"num_object_copies":180038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":90019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53164,"num_write_kb":212650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6275091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90101,"ondisk_log_size":90101,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1199712,"kb_used_data":727084,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281915808,"statfs":{"total":289910292480,"available":288681787392,"internally_reserved":0,"allocated":744534016,"data_stored":743715947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":483904513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,8,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-323,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5614,"num_write_kb":22456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008860"},"pg_stats":[{"pgid":"2.f","version":"43'5691","reported_seq":5765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.895802+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:08.895802+0000","last_peered":"2024-09-11T05:15:08.895802+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:08.895802+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5691,"log_dups_size":0,"ondisk_log_size":5691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5691,"num_write_kb":22764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2016},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5712","reported_seq":5809,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:10.649322+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:10.649322+0000","last_peered":"2024-09-11T05:15:10.649322+0000","last_clean":"2024-09-11T05:15:10.649322+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:10.649322+0000","last_undegraded":"2024-09-11T05:15:10.649322+0000","last_fullsized":"2024-09-11T05:15:10.649322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5712,"log_dups_size":0,"ondisk_log_size":5712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5618","reported_seq":5738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.699400+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:10.699400+0000","last_peered":"2024-09-11T05:15:10.699400+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:10.699400+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4021,"num_write_kb":16084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5752","reported_seq":5818,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:10.634469+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:10.634469+0000","last_peered":"2024-09-11T05:15:10.634469+0000","last_clean":"2024-09-11T05:15:10.634469+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:10.634469+0000","last_undegraded":"2024-09-11T05:15:10.634469+0000","last_fullsized":"2024-09-11T05:15:10.634469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5752,"log_dups_size":0,"ondisk_log_size":5752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23560192,"num_objects":5752,"num_object_clones":0,"num_object_copies":11504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5752,"num_write_kb":23008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5704","reported_seq":5829,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:08.902338+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:08.902338+0000","last_peered":"2024-09-11T05:15:08.902338+0000","last_clean":"2024-09-11T05:15:08.902338+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:08.902338+0000","last_undegraded":"2024-09-11T05:15:08.902338+0000","last_fullsized":"2024-09-11T05:15:08.902338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2538,"num_write_kb":10152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5618","reported_seq":5762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.873280+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:08.873280+0000","last_peered":"2024-09-11T05:15:08.873280+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:08.873280+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990848,"num_objects":5613,"num_object_clones":0,"num_object_copies":11226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2013},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5676","reported_seq":5800,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.870421+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:08.870421+0000","last_peered":"2024-09-11T05:15:08.870421+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:08.870421+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5676,"log_dups_size":0,"ondisk_log_size":5676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2841,"num_write_kb":11364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2094},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5554","reported_seq":5673,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:08.760425+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:08.760425+0000","last_peered":"2024-09-11T05:15:08.760425+0000","last_clean":"2024-09-11T05:15:08.760425+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:08.760425+0000","last_undegraded":"2024-09-11T05:15:08.760425+0000","last_fullsized":"2024-09-11T05:15:08.760425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5554,"log_dups_size":0,"ondisk_log_size":5554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2857,"num_write_kb":11428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5681","reported_seq":5807,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.709867+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:10.709867+0000","last_peered":"2024-09-11T05:15:10.709867+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:10.709867+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5681,"log_dups_size":0,"ondisk_log_size":5681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23240704,"num_objects":5674,"num_object_clones":0,"num_object_copies":11348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3749,"num_write_kb":14996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5507","reported_seq":5667,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.889584+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:08.889584+0000","last_peered":"2024-09-11T05:15:08.889584+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:08.889584+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22523904,"num_objects":5499,"num_object_clones":0,"num_object_copies":10998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2260,"num_write_kb":9040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2057},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5702","reported_seq":5814,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.607950+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:10.607950+0000","last_peered":"2024-09-11T05:15:10.607950+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:10.607950+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3731,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2175}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5594","reported_seq":8768,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:10.710944+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:10.710944+0000","last_peered":"2024-09-11T05:15:10.710944+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:10.710944+0000","last_undegraded":"2024-09-11T05:15:10.710944+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3497,"num_write_kb":13988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1531,"num_bytes_recovered":6270976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1947},{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5738","reported_seq":5864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.863125+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:08.863125+0000","last_peered":"2024-09-11T05:15:08.863125+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:08.863125+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23494656,"num_objects":5736,"num_object_clones":0,"num_object_copies":11472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2771,"num_write_kb":11084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2113},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5502","reported_seq":5630,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.847801+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:08.847801+0000","last_peered":"2024-09-11T05:15:08.847801+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:08.847801+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22503424,"num_objects":5494,"num_object_clones":0,"num_object_copies":10988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2368,"num_write_kb":9472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1999},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5532","reported_seq":5662,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.887665+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:08.887665+0000","last_peered":"2024-09-11T05:15:08.887665+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:08.887665+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1984},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5517","reported_seq":5656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:08.884554+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:08.884554+0000","last_peered":"2024-09-11T05:15:08.884554+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:08.884554+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22549097,"num_objects":5506,"num_object_clones":0,"num_object_copies":11012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2524,"num_write_kb":10096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1991},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":368706153,"num_objects":90017,"num_object_clones":0,"num_object_copies":180034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":90017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53162,"num_write_kb":212648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":6275072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":746872832,"data_stored":746785857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90098,"ondisk_log_size":90098,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453128,"kb_used_data":295096,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93918712,"statfs":{"total":96636764160,"available":96172761088,"internally_reserved":0,"allocated":302178304,"data_stored":301933845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":238048,"kb_used_data":107788,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94133792,"statfs":{"total":96636764160,"available":96393003008,"internally_reserved":0,"allocated":110374912,"data_stored":110049838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215136,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508536,"kb_used_data":324200,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93863304,"statfs":{"total":96636764160,"available":96116023296,"internally_reserved":0,"allocated":331980800,"data_stored":331732264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":333193216,"data_stored":333189737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":111042560,"data_stored":110962543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302637056,"data_stored":302633577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5691", 'reported_seq': 5765, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.895802+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:08.895802+0000', 'last_peered': '2024-09-11T05:15:08.895802+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:08.895802+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 5691, 'log_dups_size': 0, 'ondisk_log_size': 5691, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23310336, 'num_objects': 5691, 'num_object_clones': 0, 'num_object_copies': 11382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5691, 'num_write_kb': 22764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2016}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5618", 'reported_seq': 5738, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:10.699400+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:10.699400+0000', 'last_peered': '2024-09-11T05:15:10.699400+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:10.699400+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 5618, 'log_dups_size': 0, 'ondisk_log_size': 5618, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22994944, 'num_objects': 5614, 'num_object_clones': 0, 'num_object_copies': 11228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4021, 'num_write_kb': 16084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 2120}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5618", 'reported_seq': 5762, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.873280+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:08.873280+0000', 'last_peered': '2024-09-11T05:15:08.873280+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:08.873280+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 5618, 'log_dups_size': 0, 'ondisk_log_size': 5618, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22990848, 'num_objects': 5613, 'num_object_clones': 0, 'num_object_copies': 11226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5613, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2162, 'num_write_kb': 8648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2013}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:13.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5676", 'reported_seq': 5800, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.870421+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:08.870421+0000', 'last_peered': '2024-09-11T05:15:08.870421+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:08.870421+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 5676, 'log_dups_size': 0, 'ondisk_log_size': 5676, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23224320, 'num_objects': 5670, 'num_object_clones': 0, 'num_object_copies': 11340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5670, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2841, 'num_write_kb': 11364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2094}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5681", 'reported_seq': 5807, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:10.709867+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:10.709867+0000', 'last_peered': '2024-09-11T05:15:10.709867+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:10.709867+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 5681, 'log_dups_size': 0, 'ondisk_log_size': 5681, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23240704, 'num_objects': 5674, 'num_object_clones': 0, 'num_object_copies': 11348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5674, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3749, 'num_write_kb': 14996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2149}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5507", 'reported_seq': 5667, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.889584+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:08.889584+0000', 'last_peered': '2024-09-11T05:15:08.889584+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:08.889584+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 5507, 'log_dups_size': 0, 'ondisk_log_size': 5507, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22523904, 'num_objects': 5499, 'num_object_clones': 0, 'num_object_copies': 10998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5499, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2260, 'num_write_kb': 9040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2057}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5702", 'reported_seq': 5814, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:10.607950+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:10.607950+0000', 'last_peered': '2024-09-11T05:15:10.607950+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:10.607950+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 5702, 'log_dups_size': 0, 'ondisk_log_size': 5702, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23334912, 'num_objects': 5697, 'num_object_clones': 0, 'num_object_copies': 11394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3731, 'num_write_kb': 14924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 2175}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5594", 'reported_seq': 8768, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:10.710944+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:10.710944+0000', 'last_peered': '2024-09-11T05:15:10.710944+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:10.710944+0000', 'last_undegraded': '2024-09-11T05:15:10.710944+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 5594, 'log_dups_size': 0, 'ondisk_log_size': 5594, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22888448, 'num_objects': 5588, 'num_object_clones': 0, 'num_object_copies': 11176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1947, 'num_objects_unfound': 0, 'num_objects_dirty': 5588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3497, 'num_write_kb': 13988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1531, 'num_bytes_recovered': 6270976, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1947}, {'shards': '2', 'objects': 3641}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:13.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5738", 'reported_seq': 5864, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.863125+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:08.863125+0000', 'last_peered': '2024-09-11T05:15:08.863125+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:08.863125+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 5738, 'log_dups_size': 0, 'ondisk_log_size': 5738, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23494656, 'num_objects': 5736, 'num_object_clones': 0, 'num_object_copies': 11472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2771, 'num_write_kb': 11084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2113}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:13.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5502", 'reported_seq': 5630, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.847801+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:08.847801+0000', 'last_peered': '2024-09-11T05:15:08.847801+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:08.847801+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 5502, 'log_dups_size': 0, 'ondisk_log_size': 5502, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22503424, 'num_objects': 5494, 'num_object_clones': 0, 'num_object_copies': 10988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2368, 'num_write_kb': 9472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1999}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:13.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5532", 'reported_seq': 5662, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.887665+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:08.887665+0000', 'last_peered': '2024-09-11T05:15:08.887665+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:08.887665+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 5532, 'log_dups_size': 0, 'ondisk_log_size': 5532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22642688, 'num_objects': 5528, 'num_object_clones': 0, 'num_object_copies': 11056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2290, 'num_write_kb': 9160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1984}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:13.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5517", 'reported_seq': 5656, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:08.884554+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:08.884554+0000', 'last_peered': '2024-09-11T05:15:08.884554+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:08.884554+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 5517, 'log_dups_size': 0, 'ondisk_log_size': 5517, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22549097, 'num_objects': 5506, 'num_object_clones': 0, 'num_object_copies': 11012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2524, 'num_write_kb': 10096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1991}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:13.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:14.108 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 101 20 92533 92513 3.57645 2.08984 0.0617014 0.0218339 2024-09-11T05:15:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:14.690 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/180038 objects degraded (21.679%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":192,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":90019,"data_bytes":368706172,"bytes_used":1228505088,"bytes_avail":288681787392,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":180038,"degraded_ratio":0.21679312145213789,"misplaced_objects":1947,"misplaced_total":180038,"misplaced_ratio":0.010814383630122527,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":110169,"recovering_keys_per_sec":0,"num_objects_recovered":323,"num_bytes_recovered":1323008,"num_keys_recovered":0,"write_bytes_sec":1914831,"read_op_per_sec":0,"write_op_per_sec":467},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:15:14.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:14.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:15.109 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 102 20 93079 93059 3.56228 2.13281 0.00978314 0.0219211 2024-09-11T05:15:15.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:15.648 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:15.728 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-11T05:15:15.395677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376013436,"num_objects":91803,"num_object_clones":0,"num_object_copies":183606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":91803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54948,"num_write_kb":219786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6275091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91885,"ondisk_log_size":91885,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1199936,"kb_used_data":735628,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281915584,"statfs":{"total":289910292480,"available":288681558016,"internally_reserved":0,"allocated":753283072,"data_stored":752465003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":475384833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,8,7,5,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30302208,"num_objects":7398,"num_object_clones":0,"num_object_copies":14796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-323,"num_objects_unfound":0,"num_objects_dirty":7398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7398,"num_write_kb":29592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008776"},"pg_stats":[{"pgid":"2.f","version":"43'5870","reported_seq":5943,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.895065+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:13.895065+0000","last_peered":"2024-09-11T05:15:13.895065+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:13.895065+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5870,"log_dups_size":0,"ondisk_log_size":5870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24043520,"num_objects":5870,"num_object_clones":0,"num_object_copies":11740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5870,"num_write_kb":23480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2195},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5712","reported_seq":5809,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:10.649322+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:10.649322+0000","last_peered":"2024-09-11T05:15:10.649322+0000","last_clean":"2024-09-11T05:15:10.649322+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:10.649322+0000","last_undegraded":"2024-09-11T05:15:10.649322+0000","last_fullsized":"2024-09-11T05:15:10.649322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5712,"log_dups_size":0,"ondisk_log_size":5712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5618","reported_seq":5738,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.699400+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:10.699400+0000","last_peered":"2024-09-11T05:15:10.699400+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:10.699400+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5618,"log_dups_size":0,"ondisk_log_size":5618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4021,"num_write_kb":16084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2120}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5752","reported_seq":5818,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:10.634469+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:10.634469+0000","last_peered":"2024-09-11T05:15:10.634469+0000","last_clean":"2024-09-11T05:15:10.634469+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:10.634469+0000","last_undegraded":"2024-09-11T05:15:10.634469+0000","last_fullsized":"2024-09-11T05:15:10.634469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5752,"log_dups_size":0,"ondisk_log_size":5752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23560192,"num_objects":5752,"num_object_clones":0,"num_object_copies":11504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5752,"num_write_kb":23008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5894","reported_seq":6019,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:13.800339+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:13.800339+0000","last_peered":"2024-09-11T05:15:13.800339+0000","last_clean":"2024-09-11T05:15:13.800339+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:13.800339+0000","last_undegraded":"2024-09-11T05:15:13.800339+0000","last_fullsized":"2024-09-11T05:15:13.800339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":5894,"log_dups_size":0,"ondisk_log_size":5894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2728,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5802","reported_seq":5945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.909435+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:13.909435+0000","last_peered":"2024-09-11T05:15:13.909435+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:13.909435+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5802,"log_dups_size":0,"ondisk_log_size":5802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23744512,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2346,"num_write_kb":9384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2197},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5855","reported_seq":5983,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.900865+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:13.900865+0000","last_peered":"2024-09-11T05:15:13.900865+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:13.900865+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":5855,"log_dups_size":0,"ondisk_log_size":5855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23957504,"num_objects":5849,"num_object_clones":0,"num_object_copies":11698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3020,"num_write_kb":12080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2273},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5730","reported_seq":5849,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:13.907296+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:13.907296+0000","last_peered":"2024-09-11T05:15:13.907296+0000","last_clean":"2024-09-11T05:15:13.907296+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:13.907296+0000","last_undegraded":"2024-09-11T05:15:13.907296+0000","last_fullsized":"2024-09-11T05:15:13.907296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":5728,"num_object_clones":0,"num_object_copies":11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5681","reported_seq":5807,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.709867+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:10.709867+0000","last_peered":"2024-09-11T05:15:10.709867+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:10.709867+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5681,"log_dups_size":0,"ondisk_log_size":5681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23240704,"num_objects":5674,"num_object_clones":0,"num_object_copies":11348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3749,"num_write_kb":14996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5675","reported_seq":5835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.882021+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:13.882021+0000","last_peered":"2024-09-11T05:15:13.882021+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:13.882021+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5675,"log_dups_size":0,"ondisk_log_size":5675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23212032,"num_objects":5667,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2428,"num_write_kb":9712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2225},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5702","reported_seq":5814,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:10.607950+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:10.607950+0000","last_peered":"2024-09-11T05:15:10.607950+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:10.607950+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3731,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2175}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5594","reported_seq":8768,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:10.710944+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:10.710944+0000","last_peered":"2024-09-11T05:15:10.710944+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:10.710944+0000","last_undegraded":"2024-09-11T05:15:10.710944+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22888448,"num_objects":5588,"num_object_clones":0,"num_object_copies":11176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":5588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3497,"num_write_kb":13988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1531,"num_bytes_recovered":6270976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1947},{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5909","reported_seq":6033,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.937898+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:13.937898+0000","last_peered":"2024-09-11T05:15:13.937898+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:13.937898+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":5909,"log_dups_size":0,"ondisk_log_size":5909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24195072,"num_objects":5907,"num_object_clones":0,"num_object_copies":11814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2942,"num_write_kb":11768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5704","reported_seq":5833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.919501+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:13.919501+0000","last_peered":"2024-09-11T05:15:13.919501+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:13.919501+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23330816,"num_objects":5696,"num_object_clones":0,"num_object_copies":11392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2570,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2201},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5710","reported_seq":5840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.919640+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:13.919640+0000","last_peered":"2024-09-11T05:15:13.919640+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:13.919640+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5710,"log_dups_size":0,"ondisk_log_size":5710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2468,"num_write_kb":9872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2162},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5674","reported_seq":5813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:13.901629+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:13.901629+0000","last_peered":"2024-09-11T05:15:13.901629+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:13.901629+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5674,"log_dups_size":0,"ondisk_log_size":5674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23192169,"num_objects":5663,"num_object_clones":0,"num_object_copies":11326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2148},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376013417,"num_objects":91801,"num_object_clones":0,"num_object_copies":183602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1947,"num_objects_unfound":0,"num_objects_dirty":91801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54946,"num_write_kb":219784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1532,"num_bytes_recovered":6275072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":755556352,"data_stored":755469377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91882,"ondisk_log_size":91882,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453128,"kb_used_data":295096,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93918712,"statfs":{"total":96636764160,"available":96172761088,"internally_reserved":0,"allocated":302178304,"data_stored":301933845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":238048,"kb_used_data":107788,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94133792,"statfs":{"total":96636764160,"available":96393003008,"internally_reserved":0,"allocated":110374912,"data_stored":110049838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215137,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508760,"kb_used_data":332744,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93863080,"statfs":{"total":96636764160,"available":96115793920,"internally_reserved":0,"allocated":340729856,"data_stored":340481320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,1,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":341876736,"data_stored":341873257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":111042560,"data_stored":110962543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302637056,"data_stored":302633577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:16.109 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 103 20 93634 93614 3.54875 2.16797 0.0707755 0.0220063 2024-09-11T05:15:17.109 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 104 20 94210 94190 3.53626 2.25 0.0267004 0.0220837 2024-09-11T05:15:18.110 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 105 20 94788 94768 3.52406 2.25781 0.0132995 0.0221596 2024-09-11T05:15:18.730 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:19.110 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 106 20 95341 95321 3.51119 2.16016 0.0199172 0.0222389 2024-09-11T05:15:19.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:19.652 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:19.740 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-11T05:15:19.396708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":387392124,"num_objects":94581,"num_object_clones":0,"num_object_copies":189162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1782,"num_objects_unfound":0,"num_objects_dirty":94581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57726,"num_write_kb":230898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1698,"num_bytes_recovered":6950931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94663,"ondisk_log_size":94663,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222800,"kb_used_data":758496,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281892720,"statfs":{"total":289910292480,"available":288658145280,"internally_reserved":0,"allocated":776699904,"data_stored":775881835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":475384833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,9,3,7,5,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29999104,"num_objects":7324,"num_object_clones":0,"num_object_copies":14648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-326,"num_objects_unfound":0,"num_objects_dirty":7324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7324,"num_write_kb":29296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":1335296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008882"},"pg_stats":[{"pgid":"2.f","version":"43'6039","reported_seq":6113,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.939097+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:18.939097+0000","last_peered":"2024-09-11T05:15:18.939097+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:18.939097+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24735744,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6039,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2364},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5869","reported_seq":5966,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:15.685810+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:15.685810+0000","last_peered":"2024-09-11T05:15:15.685810+0000","last_clean":"2024-09-11T05:15:15.685810+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:15.685810+0000","last_undegraded":"2024-09-11T05:15:15.685810+0000","last_fullsized":"2024-09-11T05:15:15.685810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4267,"num_write_kb":17068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5782","reported_seq":5902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:15.451354+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:15.451354+0000","last_peered":"2024-09-11T05:15:15.451354+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:15.451354+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5782,"log_dups_size":0,"ondisk_log_size":5782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4185,"num_write_kb":16740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5924","reported_seq":5990,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:15.703476+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:15.703476+0000","last_peered":"2024-09-11T05:15:15.703476+0000","last_clean":"2024-09-11T05:15:15.703476+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:15.703476+0000","last_undegraded":"2024-09-11T05:15:15.703476+0000","last_fullsized":"2024-09-11T05:15:15.703476+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5924,"log_dups_size":0,"ondisk_log_size":5924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5924,"num_write_kb":23696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6060","reported_seq":6183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:18.943263+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:18.943263+0000","last_peered":"2024-09-11T05:15:18.943263+0000","last_clean":"2024-09-11T05:15:18.943263+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:18.943263+0000","last_undegraded":"2024-09-11T05:15:18.943263+0000","last_fullsized":"2024-09-11T05:15:18.943263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6060,"log_dups_size":0,"ondisk_log_size":6060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2894,"num_write_kb":11576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5972","reported_seq":6115,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.954488+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:18.954488+0000","last_peered":"2024-09-11T05:15:18.954488+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:18.954488+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5972,"log_dups_size":0,"ondisk_log_size":5972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24440832,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6041","reported_seq":6169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.954497+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:18.954497+0000","last_peered":"2024-09-11T05:15:18.954497+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:18.954497+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6041,"log_dups_size":0,"ondisk_log_size":6041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24719360,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3206,"num_write_kb":12824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5901","reported_seq":6020,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:18.913366+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:18.913366+0000","last_peered":"2024-09-11T05:15:18.913366+0000","last_clean":"2024-09-11T05:15:18.913366+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:18.913366+0000","last_undegraded":"2024-09-11T05:15:18.913366+0000","last_fullsized":"2024-09-11T05:15:18.913366+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162304,"num_objects":5899,"num_object_clones":0,"num_object_copies":11798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3204,"num_write_kb":12816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5859","reported_seq":5985,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:15.668418+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:15.668418+0000","last_peered":"2024-09-11T05:15:15.668418+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:15.668418+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23969792,"num_objects":5852,"num_object_clones":0,"num_object_copies":11704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3927,"num_write_kb":15708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5854","reported_seq":6013,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.932400+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:18.932400+0000","last_peered":"2024-09-11T05:15:18.932400+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:18.932400+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5854,"log_dups_size":0,"ondisk_log_size":5854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2607,"num_write_kb":10428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2404},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5888","reported_seq":6000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:15.702841+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:15.702841+0000","last_peered":"2024-09-11T05:15:15.702841+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:15.702841+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3917,"num_write_kb":15668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2361}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5761","reported_seq":9265,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:15.706197+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:15.706197+0000","last_peered":"2024-09-11T05:15:15.706197+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:15.706197+0000","last_undegraded":"2024-09-11T05:15:15.706197+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5761,"log_dups_size":0,"ondisk_log_size":5761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23572480,"num_objects":5755,"num_object_clones":0,"num_object_copies":11510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1782,"num_objects_unfound":0,"num_objects_dirty":5755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":6946816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1782},{"shards":"2","objects":3973}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6070","reported_seq":6197,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.863682+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:18.863682+0000","last_peered":"2024-09-11T05:15:18.863682+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:18.863682+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6070,"log_dups_size":0,"ondisk_log_size":6070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24854528,"num_objects":6068,"num_object_clones":0,"num_object_copies":12136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3103,"num_write_kb":12412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2445},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5883","reported_seq":6012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.975155+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:18.975155+0000","last_peered":"2024-09-11T05:15:18.975155+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:18.975155+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24064000,"num_objects":5875,"num_object_clones":0,"num_object_copies":11750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2749,"num_write_kb":10996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2380},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5895","reported_seq":6024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.983733+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:18.983733+0000","last_peered":"2024-09-11T05:15:18.983733+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:18.983733+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5895,"log_dups_size":0,"ondisk_log_size":5895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24129536,"num_objects":5891,"num_object_clones":0,"num_object_copies":11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2653,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2347},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5862","reported_seq":6001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.869843+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:18.869843+0000","last_peered":"2024-09-11T05:15:18.869843+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:18.869843+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5862,"log_dups_size":0,"ondisk_log_size":5862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23962217,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2336},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":387392105,"num_objects":94579,"num_object_clones":0,"num_object_copies":189158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1782,"num_objects_unfound":0,"num_objects_dirty":94579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57724,"num_write_kb":230896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1697,"num_bytes_recovered":6950912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":779014144,"data_stored":778927169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94660,"ondisk_log_size":94660,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462808,"kb_used_data":300616,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93909032,"statfs":{"total":96636764160,"available":96162848768,"internally_reserved":0,"allocated":307830784,"data_stored":307586325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":166063787},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":238656,"kb_used_data":116720,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94133184,"statfs":{"total":96636764160,"available":96392380416,"internally_reserved":0,"allocated":119521280,"data_stored":119196206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215138,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521336,"kb_used_data":341160,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93850504,"statfs":{"total":96636764160,"available":96102916096,"internally_reserved":0,"allocated":349347840,"data_stored":349099304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350572544,"data_stored":350569065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":120229888,"data_stored":120149871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308211712,"data_stored":308208233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:19.741 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:20.113 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 107 20 95880 95860 3.49803 2.10547 0.0260064 0.0223252 2024-09-11T05:15:20.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:20.535 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:20.571 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-11T05:15:19.396708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":387392124,"num_objects":94581,"num_object_clones":0,"num_object_copies":189162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1782,"num_objects_unfound":0,"num_objects_dirty":94581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57726,"num_write_kb":230898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1698,"num_bytes_recovered":6950931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94663,"ondisk_log_size":94663,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222800,"kb_used_data":758496,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281892720,"statfs":{"total":289910292480,"available":288658145280,"internally_reserved":0,"allocated":776699904,"data_stored":775881835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":475384833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,9,3,7,5,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29999104,"num_objects":7324,"num_object_clones":0,"num_object_copies":14648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-326,"num_objects_unfound":0,"num_objects_dirty":7324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7324,"num_write_kb":29296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":1335296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008882"},"pg_stats":[{"pgid":"2.f","version":"43'6039","reported_seq":6113,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.939097+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:18.939097+0000","last_peered":"2024-09-11T05:15:18.939097+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:18.939097+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24735744,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6039,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2364},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5869","reported_seq":5966,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:15.685810+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:15.685810+0000","last_peered":"2024-09-11T05:15:15.685810+0000","last_clean":"2024-09-11T05:15:15.685810+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:15.685810+0000","last_undegraded":"2024-09-11T05:15:15.685810+0000","last_fullsized":"2024-09-11T05:15:15.685810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4267,"num_write_kb":17068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5782","reported_seq":5902,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:15.451354+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:15.451354+0000","last_peered":"2024-09-11T05:15:15.451354+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:15.451354+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5782,"log_dups_size":0,"ondisk_log_size":5782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4185,"num_write_kb":16740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5924","reported_seq":5990,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:15.703476+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:15.703476+0000","last_peered":"2024-09-11T05:15:15.703476+0000","last_clean":"2024-09-11T05:15:15.703476+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:15.703476+0000","last_undegraded":"2024-09-11T05:15:15.703476+0000","last_fullsized":"2024-09-11T05:15:15.703476+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":5924,"log_dups_size":0,"ondisk_log_size":5924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5924,"num_write_kb":23696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6060","reported_seq":6183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:18.943263+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:18.943263+0000","last_peered":"2024-09-11T05:15:18.943263+0000","last_clean":"2024-09-11T05:15:18.943263+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:18.943263+0000","last_undegraded":"2024-09-11T05:15:18.943263+0000","last_fullsized":"2024-09-11T05:15:18.943263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6060,"log_dups_size":0,"ondisk_log_size":6060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2894,"num_write_kb":11576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5972","reported_seq":6115,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.954488+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:18.954488+0000","last_peered":"2024-09-11T05:15:18.954488+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:18.954488+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5972,"log_dups_size":0,"ondisk_log_size":5972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24440832,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6041","reported_seq":6169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.954497+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:18.954497+0000","last_peered":"2024-09-11T05:15:18.954497+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:18.954497+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6041,"log_dups_size":0,"ondisk_log_size":6041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24719360,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3206,"num_write_kb":12824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5901","reported_seq":6020,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:18.913366+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:18.913366+0000","last_peered":"2024-09-11T05:15:18.913366+0000","last_clean":"2024-09-11T05:15:18.913366+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:18.913366+0000","last_undegraded":"2024-09-11T05:15:18.913366+0000","last_fullsized":"2024-09-11T05:15:18.913366+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162304,"num_objects":5899,"num_object_clones":0,"num_object_copies":11798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3204,"num_write_kb":12816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5859","reported_seq":5985,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:15.668418+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:15.668418+0000","last_peered":"2024-09-11T05:15:15.668418+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:15.668418+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23969792,"num_objects":5852,"num_object_clones":0,"num_object_copies":11704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3927,"num_write_kb":15708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741142+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741142+0000","last_peered":"2024-09-11T05:14:13.741142+0000","last_clean":"2024-09-11T05:14:13.741142+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741142+0000","last_undegraded":"2024-09-11T05:14:13.741142+0000","last_fullsized":"2024-09-11T05:14:13.741142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5854","reported_seq":6013,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.932400+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:18.932400+0000","last_peered":"2024-09-11T05:15:18.932400+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:18.932400+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5854,"log_dups_size":0,"ondisk_log_size":5854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2607,"num_write_kb":10428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2404},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5888","reported_seq":6000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:15.702841+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:15.702841+0000","last_peered":"2024-09-11T05:15:15.702841+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:15.702841+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3917,"num_write_kb":15668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2361}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5761","reported_seq":9265,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:15.706197+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:15.706197+0000","last_peered":"2024-09-11T05:15:15.706197+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:15.706197+0000","last_undegraded":"2024-09-11T05:15:15.706197+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5761,"log_dups_size":0,"ondisk_log_size":5761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23572480,"num_objects":5755,"num_object_clones":0,"num_object_copies":11510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1782,"num_objects_unfound":0,"num_objects_dirty":5755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":6946816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1782},{"shards":"2","objects":3973}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6070","reported_seq":6197,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.863682+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:18.863682+0000","last_peered":"2024-09-11T05:15:18.863682+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:18.863682+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6070,"log_dups_size":0,"ondisk_log_size":6070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24854528,"num_objects":6068,"num_object_clones":0,"num_object_copies":12136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3103,"num_write_kb":12412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2445},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5883","reported_seq":6012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.975155+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:18.975155+0000","last_peered":"2024-09-11T05:15:18.975155+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:18.975155+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24064000,"num_objects":5875,"num_object_clones":0,"num_object_copies":11750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2749,"num_write_kb":10996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2380},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738887+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738887+0000","last_peered":"2024-09-11T05:14:13.738887+0000","last_clean":"2024-09-11T05:14:13.738887+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738887+0000","last_undegraded":"2024-09-11T05:14:13.738887+0000","last_fullsized":"2024-09-11T05:14:13.738887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5895","reported_seq":6024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.983733+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:18.983733+0000","last_peered":"2024-09-11T05:15:18.983733+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:18.983733+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5895,"log_dups_size":0,"ondisk_log_size":5895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24129536,"num_objects":5891,"num_object_clones":0,"num_object_copies":11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2653,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2347},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5862","reported_seq":6001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.869843+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:18.869843+0000","last_peered":"2024-09-11T05:15:18.869843+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:18.869843+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5862,"log_dups_size":0,"ondisk_log_size":5862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23962217,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2336},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":387392105,"num_objects":94579,"num_object_clones":0,"num_object_copies":189158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1782,"num_objects_unfound":0,"num_objects_dirty":94579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57724,"num_write_kb":230896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1697,"num_bytes_recovered":6950912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":779014144,"data_stored":778927169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94660,"ondisk_log_size":94660,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462808,"kb_used_data":300616,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93909032,"statfs":{"total":96636764160,"available":96162848768,"internally_reserved":0,"allocated":307830784,"data_stored":307586325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":166063787},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":238656,"kb_used_data":116720,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94133184,"statfs":{"total":96636764160,"available":96392380416,"internally_reserved":0,"allocated":119521280,"data_stored":119196206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":124841643},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215138,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521336,"kb_used_data":341160,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93850504,"statfs":{"total":96636764160,"available":96102916096,"internally_reserved":0,"allocated":349347840,"data_stored":349099304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350572544,"data_stored":350569065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":120229888,"data_stored":120149871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308211712,"data_stored":308208233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:20.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6039", 'reported_seq': 6113, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.939097+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:18.939097+0000', 'last_peered': '2024-09-11T05:15:18.939097+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:18.939097+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 6039, 'log_dups_size': 0, 'ondisk_log_size': 6039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24735744, 'num_objects': 6039, 'num_object_clones': 0, 'num_object_copies': 12078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6039, 'num_write_kb': 24156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2364}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5782", 'reported_seq': 5902, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:15.451354+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:15.451354+0000', 'last_peered': '2024-09-11T05:15:15.451354+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:15.451354+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 5782, 'log_dups_size': 0, 'ondisk_log_size': 5782, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23666688, 'num_objects': 5778, 'num_object_clones': 0, 'num_object_copies': 11556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4185, 'num_write_kb': 16740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 2284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5972", 'reported_seq': 6115, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.954488+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:18.954488+0000', 'last_peered': '2024-09-11T05:15:18.954488+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:18.954488+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 5972, 'log_dups_size': 0, 'ondisk_log_size': 5972, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24440832, 'num_objects': 5967, 'num_object_clones': 0, 'num_object_copies': 11934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2516, 'num_write_kb': 10064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2367}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6041", 'reported_seq': 6169, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.954497+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:18.954497+0000', 'last_peered': '2024-09-11T05:15:18.954497+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:18.954497+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 6041, 'log_dups_size': 0, 'ondisk_log_size': 6041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24719360, 'num_objects': 6035, 'num_object_clones': 0, 'num_object_copies': 12070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3206, 'num_write_kb': 12824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2459}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:20.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5859", 'reported_seq': 5985, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:15.668418+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:15.668418+0000', 'last_peered': '2024-09-11T05:15:15.668418+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:15.668418+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 5859, 'log_dups_size': 0, 'ondisk_log_size': 5859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23969792, 'num_objects': 5852, 'num_object_clones': 0, 'num_object_copies': 11704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5852, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3927, 'num_write_kb': 15708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2327}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5854", 'reported_seq': 6013, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.932400+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:18.932400+0000', 'last_peered': '2024-09-11T05:15:18.932400+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:18.932400+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 5854, 'log_dups_size': 0, 'ondisk_log_size': 5854, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23945216, 'num_objects': 5846, 'num_object_clones': 0, 'num_object_copies': 11692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2607, 'num_write_kb': 10428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2404}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5888", 'reported_seq': 6000, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:15.702841+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:15.702841+0000', 'last_peered': '2024-09-11T05:15:15.702841+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:15.702841+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 5888, 'log_dups_size': 0, 'ondisk_log_size': 5888, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24096768, 'num_objects': 5883, 'num_object_clones': 0, 'num_object_copies': 11766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5883, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3917, 'num_write_kb': 15668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 2361}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5761", 'reported_seq': 9265, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:15.706197+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:15.706197+0000', 'last_peered': '2024-09-11T05:15:15.706197+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:15.706197+0000', 'last_undegraded': '2024-09-11T05:15:15.706197+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 5761, 'log_dups_size': 0, 'ondisk_log_size': 5761, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23572480, 'num_objects': 5755, 'num_object_clones': 0, 'num_object_copies': 11510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1782, 'num_objects_unfound': 0, 'num_objects_dirty': 5755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3664, 'num_write_kb': 14656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1696, 'num_bytes_recovered': 6946816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1782}, {'shards': '2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6070", 'reported_seq': 6197, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.863682+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:18.863682+0000', 'last_peered': '2024-09-11T05:15:18.863682+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:18.863682+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 6070, 'log_dups_size': 0, 'ondisk_log_size': 6070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24854528, 'num_objects': 6068, 'num_object_clones': 0, 'num_object_copies': 12136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3103, 'num_write_kb': 12412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2445}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:20.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5883", 'reported_seq': 6012, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.975155+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:18.975155+0000', 'last_peered': '2024-09-11T05:15:18.975155+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:18.975155+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 5883, 'log_dups_size': 0, 'ondisk_log_size': 5883, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24064000, 'num_objects': 5875, 'num_object_clones': 0, 'num_object_copies': 11750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5875, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2749, 'num_write_kb': 10996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2380}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:20.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5895", 'reported_seq': 6024, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.983733+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:18.983733+0000', 'last_peered': '2024-09-11T05:15:18.983733+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:18.983733+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 5895, 'log_dups_size': 0, 'ondisk_log_size': 5895, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24129536, 'num_objects': 5891, 'num_object_clones': 0, 'num_object_copies': 11782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2653, 'num_write_kb': 10612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2347}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:20.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5862", 'reported_seq': 6001, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:18.869843+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:18.869843+0000', 'last_peered': '2024-09-11T05:15:18.869843+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:18.869843+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 5862, 'log_dups_size': 0, 'ondisk_log_size': 5862, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23962217, 'num_objects': 5851, 'num_object_clones': 0, 'num_object_copies': 11702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2869, 'num_write_kb': 11476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2336}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:20.575 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:21.111 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 108 20 96397 96377 3.48435 2.01953 0.0619403 0.0224123 2024-09-11T05:15:21.494 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:21.495 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/189162 objects degraded (20.634%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94581,"data_bytes":387392124,"bytes_used":1252147200,"bytes_avail":288658145280,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":189162,"degraded_ratio":0.20633636776942516,"misplaced_objects":1782,"misplaced_total":189162,"misplaced_ratio":0.0094204967170996284,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":111192,"recovering_keys_per_sec":0,"num_objects_recovered":326,"num_bytes_recovered":1335296,"num_keys_recovered":0,"write_bytes_sec":2498076,"read_op_per_sec":0,"write_op_per_sec":609},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:15:21.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:21.548 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:22.111 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 109 20 96881 96861 3.46973 1.89062 0.0417679 0.0225076 2024-09-11T05:15:22.379 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:22.507 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-11T05:15:21.397232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":391660156,"num_objects":95623,"num_object_clones":0,"num_object_copies":191246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":95623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58768,"num_write_kb":235066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":7684115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95705,"ondisk_log_size":95705,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1249772,"kb_used_data":772988,"kb_used_omap":12,"kb_used_meta":476723,"kb_avail":281865748,"statfs":{"total":289910292480,"available":288630525952,"internally_reserved":0,"allocated":791539712,"data_stored":790721643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":488164353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,4,10,8,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27160576,"num_objects":6631,"num_object_clones":0,"num_object_copies":13262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-505,"num_objects_unfound":0,"num_objects_dirty":6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6631,"num_write_kb":26524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":2068480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005457"},"pg_stats":[{"pgid":"2.f","version":"43'6039","reported_seq":6113,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.939097+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:18.939097+0000","last_peered":"2024-09-11T05:15:18.939097+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:18.939097+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24735744,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6039,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2364},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6045","reported_seq":6141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:20.712524+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:20.712524+0000","last_peered":"2024-09-11T05:15:20.712524+0000","last_clean":"2024-09-11T05:15:20.712524+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:20.712524+0000","last_undegraded":"2024-09-11T05:15:20.712524+0000","last_fullsized":"2024-09-11T05:15:20.712524+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6045,"log_dups_size":0,"ondisk_log_size":6045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24735744,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":17772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5964","reported_seq":6084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.675218+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:20.675218+0000","last_peered":"2024-09-11T05:15:20.675218+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:20.675218+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4367,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6123","reported_seq":6189,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:20.734941+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:20.734941+0000","last_peered":"2024-09-11T05:15:20.734941+0000","last_clean":"2024-09-11T05:15:20.734941+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:20.734941+0000","last_undegraded":"2024-09-11T05:15:20.734941+0000","last_fullsized":"2024-09-11T05:15:20.734941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6123,"log_dups_size":0,"ondisk_log_size":6123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25079808,"num_objects":6123,"num_object_clones":0,"num_object_copies":12246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6123,"num_write_kb":24492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6060","reported_seq":6183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:18.943263+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:18.943263+0000","last_peered":"2024-09-11T05:15:18.943263+0000","last_clean":"2024-09-11T05:15:18.943263+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:18.943263+0000","last_undegraded":"2024-09-11T05:15:18.943263+0000","last_fullsized":"2024-09-11T05:15:18.943263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6060,"log_dups_size":0,"ondisk_log_size":6060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2894,"num_write_kb":11576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5972","reported_seq":6115,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.954488+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:18.954488+0000","last_peered":"2024-09-11T05:15:18.954488+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:18.954488+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":5972,"log_dups_size":0,"ondisk_log_size":5972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24440832,"num_objects":5967,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6041","reported_seq":6169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.954497+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:18.954497+0000","last_peered":"2024-09-11T05:15:18.954497+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:18.954497+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6041,"log_dups_size":0,"ondisk_log_size":6041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24719360,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3206,"num_write_kb":12824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5901","reported_seq":6020,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:18.913366+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:18.913366+0000","last_peered":"2024-09-11T05:15:18.913366+0000","last_clean":"2024-09-11T05:15:18.913366+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:18.913366+0000","last_undegraded":"2024-09-11T05:15:18.913366+0000","last_fullsized":"2024-09-11T05:15:18.913366+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162304,"num_objects":5899,"num_object_clones":0,"num_object_copies":11798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3204,"num_write_kb":12816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6022","reported_seq":6148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.699389+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:20.699389+0000","last_peered":"2024-09-11T05:15:20.699389+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:20.699389+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6022,"log_dups_size":0,"ondisk_log_size":6022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4090,"num_write_kb":16360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2490}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5854","reported_seq":6013,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.932400+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:18.932400+0000","last_peered":"2024-09-11T05:15:18.932400+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:18.932400+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":5854,"log_dups_size":0,"ondisk_log_size":5854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2607,"num_write_kb":10428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2404},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6052","reported_seq":6165,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.712070+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:20.712070+0000","last_peered":"2024-09-11T05:15:20.712070+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:20.712070+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6052,"log_dups_size":0,"ondisk_log_size":6052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4081,"num_write_kb":16324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2525}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5919","reported_seq":9780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:20.684000+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:20.684000+0000","last_peered":"2024-09-11T05:15:20.684000+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:20.684000+0000","last_undegraded":"2024-09-11T05:15:20.684000+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1875,"num_bytes_recovered":7680000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1603},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6070","reported_seq":6197,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.863682+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:18.863682+0000","last_peered":"2024-09-11T05:15:18.863682+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:18.863682+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6070,"log_dups_size":0,"ondisk_log_size":6070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24854528,"num_objects":6068,"num_object_clones":0,"num_object_copies":12136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3103,"num_write_kb":12412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2445},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5883","reported_seq":6012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.975155+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:18.975155+0000","last_peered":"2024-09-11T05:15:18.975155+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:18.975155+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24064000,"num_objects":5875,"num_object_clones":0,"num_object_copies":11750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2749,"num_write_kb":10996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2380},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5895","reported_seq":6024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.983733+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:18.983733+0000","last_peered":"2024-09-11T05:15:18.983733+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:18.983733+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":5895,"log_dups_size":0,"ondisk_log_size":5895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24129536,"num_objects":5891,"num_object_clones":0,"num_object_copies":11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2653,"num_write_kb":10612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2347},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5862","reported_seq":6001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:18.869843+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:18.869843+0000","last_peered":"2024-09-11T05:15:18.869843+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:18.869843+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":5862,"log_dups_size":0,"ondisk_log_size":5862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23962217,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2336},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":391660137,"num_objects":95621,"num_object_clones":0,"num_object_copies":191242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":95621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58766,"num_write_kb":235064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1876,"num_bytes_recovered":7684096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":793829376,"data_stored":793742401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95702,"ondisk_log_size":95702,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472516,"kb_used_data":306164,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93899324,"statfs":{"total":96636764160,"available":96152907776,"internally_reserved":0,"allocated":313511936,"data_stored":313267477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255920,"kb_used_data":125664,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94115920,"statfs":{"total":96636764160,"available":96374702080,"internally_reserved":0,"allocated":128679936,"data_stored":128354862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215138,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521336,"kb_used_data":341160,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93850504,"statfs":{"total":96636764160,"available":96102916096,"internally_reserved":0,"allocated":349347840,"data_stored":349099304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350572544,"data_stored":350569065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":129376256,"data_stored":129296239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313880576,"data_stored":313877097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:23.113 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 110 20 97424 97404 3.45747 2.12109 0.0333158 0.0225875 2024-09-11T05:15:24.112 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 111 20 97995 97975 3.44642 2.23047 0.0326529 0.0226614 2024-09-11T05:15:25.114 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 112 20 98548 98528 3.43485 2.16016 0.0297491 0.0227378 2024-09-11T05:15:25.509 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:26.115 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 113 20 99058 99038 3.42208 1.99219 0.0132375 0.0228191 2024-09-11T05:15:26.285 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:26.288 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:26.359 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-11T05:15:25.398283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398582396,"num_objects":97313,"num_object_clones":0,"num_object_copies":194626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":97313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60458,"num_write_kb":241826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":7684115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97395,"ondisk_log_size":97395,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1266020,"kb_used_data":780916,"kb_used_omap":12,"kb_used_meta":485043,"kb_avail":281849500,"statfs":{"total":289910292480,"available":288613888000,"internally_reserved":0,"allocated":799657984,"data_stored":798839915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":496684033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,4,10,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7294,"num_write_kb":29176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002997"},"pg_stats":[{"pgid":"2.f","version":"43'6217","reported_seq":6291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.049167+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:24.049167+0000","last_peered":"2024-09-11T05:15:24.049167+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:24.049167+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6217,"log_dups_size":0,"ondisk_log_size":6217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25464832,"num_objects":6217,"num_object_clones":0,"num_object_copies":12434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6217,"num_write_kb":24868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2542},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6045","reported_seq":6141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:20.712524+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:20.712524+0000","last_peered":"2024-09-11T05:15:20.712524+0000","last_clean":"2024-09-11T05:15:20.712524+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:20.712524+0000","last_undegraded":"2024-09-11T05:15:20.712524+0000","last_fullsized":"2024-09-11T05:15:20.712524+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6045,"log_dups_size":0,"ondisk_log_size":6045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24735744,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":17772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5964","reported_seq":6084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.675218+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:20.675218+0000","last_peered":"2024-09-11T05:15:20.675218+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:20.675218+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4367,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6123","reported_seq":6189,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:20.734941+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:20.734941+0000","last_peered":"2024-09-11T05:15:20.734941+0000","last_clean":"2024-09-11T05:15:20.734941+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:20.734941+0000","last_undegraded":"2024-09-11T05:15:20.734941+0000","last_fullsized":"2024-09-11T05:15:20.734941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6123,"log_dups_size":0,"ondisk_log_size":6123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25079808,"num_objects":6123,"num_object_clones":0,"num_object_copies":12246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6123,"num_write_kb":24492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6215","reported_seq":6340,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:23.969724+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:23.969724+0000","last_peered":"2024-09-11T05:15:23.969724+0000","last_clean":"2024-09-11T05:15:23.969724+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:23.969724+0000","last_undegraded":"2024-09-11T05:15:23.969724+0000","last_fullsized":"2024-09-11T05:15:23.969724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3049,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6134","reported_seq":6278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.931313+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:23.931313+0000","last_peered":"2024-09-11T05:15:23.931313+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:23.931313+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6134,"log_dups_size":0,"ondisk_log_size":6134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2529},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6211","reported_seq":6339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.905948+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:23.905948+0000","last_peered":"2024-09-11T05:15:23.905948+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:23.905948+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2629},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6083","reported_seq":6202,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:24.028557+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:24.028557+0000","last_peered":"2024-09-11T05:15:24.028557+0000","last_clean":"2024-09-11T05:15:24.028557+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:24.028557+0000","last_undegraded":"2024-09-11T05:15:24.028557+0000","last_fullsized":"2024-09-11T05:15:24.028557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6083,"log_dups_size":0,"ondisk_log_size":6083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24907776,"num_objects":6081,"num_object_clones":0,"num_object_copies":12162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6022","reported_seq":6148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.699389+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:20.699389+0000","last_peered":"2024-09-11T05:15:20.699389+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:20.699389+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6022,"log_dups_size":0,"ondisk_log_size":6022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4090,"num_write_kb":16360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2490}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6023","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.991249+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:23.991249+0000","last_peered":"2024-09-11T05:15:23.991249+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:23.991249+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6023,"log_dups_size":0,"ondisk_log_size":6023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6052","reported_seq":6165,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.712070+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:20.712070+0000","last_peered":"2024-09-11T05:15:20.712070+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:20.712070+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6052,"log_dups_size":0,"ondisk_log_size":6052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4081,"num_write_kb":16324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2525}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5919","reported_seq":9780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:20.684000+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:20.684000+0000","last_peered":"2024-09-11T05:15:20.684000+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:20.684000+0000","last_undegraded":"2024-09-11T05:15:20.684000+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1875,"num_bytes_recovered":7680000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1603},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6229","reported_seq":6356,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.046005+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:24.046005+0000","last_peered":"2024-09-11T05:15:24.046005+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:24.046005+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6229,"log_dups_size":0,"ondisk_log_size":6229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3262,"num_write_kb":13048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2604},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6058","reported_seq":6187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.073288+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:24.073288+0000","last_peered":"2024-09-11T05:15:24.073288+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:24.073288+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6058,"log_dups_size":0,"ondisk_log_size":6058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2924,"num_write_kb":11696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2555},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6064","reported_seq":6193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.016918+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:24.016918+0000","last_peered":"2024-09-11T05:15:24.016918+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:24.016918+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24821760,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2822,"num_write_kb":11288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6033","reported_seq":6172,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.041350+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:24.041350+0000","last_peered":"2024-09-11T05:15:24.041350+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:24.041350+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6033,"log_dups_size":0,"ondisk_log_size":6033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24662633,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2507},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398582377,"num_objects":97311,"num_object_clones":0,"num_object_copies":194622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":97311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60456,"num_write_kb":241824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1876,"num_bytes_recovered":7684096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802217984,"data_stored":802131009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97392,"ondisk_log_size":97392,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472516,"kb_used_data":306164,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93899324,"statfs":{"total":96636764160,"available":96152907776,"internally_reserved":0,"allocated":313511936,"data_stored":313267477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255920,"kb_used_data":125664,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94115920,"statfs":{"total":96636764160,"available":96374702080,"internally_reserved":0,"allocated":128679936,"data_stored":128354862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215139,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537584,"kb_used_data":349088,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93834256,"statfs":{"total":96636764160,"available":96086278144,"internally_reserved":0,"allocated":357466112,"data_stored":357217576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":192999083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358961152,"data_stored":358957673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":129376256,"data_stored":129296239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313880576,"data_stored":313877097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:26.361 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:27.115 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 114 20 99580 99560 3.40995 2.03906 0.10383 0.0228976 2024-09-11T05:15:27.154 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:27.154 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:27.211 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-11T05:15:25.398283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398582396,"num_objects":97313,"num_object_clones":0,"num_object_copies":194626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":97313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60458,"num_write_kb":241826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":7684115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97395,"ondisk_log_size":97395,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1266020,"kb_used_data":780916,"kb_used_omap":12,"kb_used_meta":485043,"kb_avail":281849500,"statfs":{"total":289910292480,"available":288613888000,"internally_reserved":0,"allocated":799657984,"data_stored":798839915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":496684033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,4,10,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7294,"num_write_kb":29176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002997"},"pg_stats":[{"pgid":"2.f","version":"43'6217","reported_seq":6291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.049167+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:24.049167+0000","last_peered":"2024-09-11T05:15:24.049167+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:24.049167+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6217,"log_dups_size":0,"ondisk_log_size":6217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25464832,"num_objects":6217,"num_object_clones":0,"num_object_copies":12434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6217,"num_write_kb":24868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2542},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6045","reported_seq":6141,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:20.712524+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:20.712524+0000","last_peered":"2024-09-11T05:15:20.712524+0000","last_clean":"2024-09-11T05:15:20.712524+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:20.712524+0000","last_undegraded":"2024-09-11T05:15:20.712524+0000","last_fullsized":"2024-09-11T05:15:20.712524+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6045,"log_dups_size":0,"ondisk_log_size":6045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24735744,"num_objects":6039,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":17772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5964","reported_seq":6084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.675218+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:20.675218+0000","last_peered":"2024-09-11T05:15:20.675218+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:20.675218+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4367,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6123","reported_seq":6189,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:20.734941+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:20.734941+0000","last_peered":"2024-09-11T05:15:20.734941+0000","last_clean":"2024-09-11T05:15:20.734941+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:20.734941+0000","last_undegraded":"2024-09-11T05:15:20.734941+0000","last_fullsized":"2024-09-11T05:15:20.734941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6123,"log_dups_size":0,"ondisk_log_size":6123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25079808,"num_objects":6123,"num_object_clones":0,"num_object_copies":12246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6123,"num_write_kb":24492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6215","reported_seq":6340,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:23.969724+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:23.969724+0000","last_peered":"2024-09-11T05:15:23.969724+0000","last_clean":"2024-09-11T05:15:23.969724+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:23.969724+0000","last_undegraded":"2024-09-11T05:15:23.969724+0000","last_fullsized":"2024-09-11T05:15:23.969724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3049,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6134","reported_seq":6278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.931313+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:23.931313+0000","last_peered":"2024-09-11T05:15:23.931313+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:23.931313+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6134,"log_dups_size":0,"ondisk_log_size":6134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2529},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6211","reported_seq":6339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.905948+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:23.905948+0000","last_peered":"2024-09-11T05:15:23.905948+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:23.905948+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2629},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6083","reported_seq":6202,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:24.028557+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:24.028557+0000","last_peered":"2024-09-11T05:15:24.028557+0000","last_clean":"2024-09-11T05:15:24.028557+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:24.028557+0000","last_undegraded":"2024-09-11T05:15:24.028557+0000","last_fullsized":"2024-09-11T05:15:24.028557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6083,"log_dups_size":0,"ondisk_log_size":6083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24907776,"num_objects":6081,"num_object_clones":0,"num_object_copies":12162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6022","reported_seq":6148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.699389+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:20.699389+0000","last_peered":"2024-09-11T05:15:20.699389+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:20.699389+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6022,"log_dups_size":0,"ondisk_log_size":6022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4090,"num_write_kb":16360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2490}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6023","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.991249+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:23.991249+0000","last_peered":"2024-09-11T05:15:23.991249+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:23.991249+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6023,"log_dups_size":0,"ondisk_log_size":6023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6052","reported_seq":6165,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:20.712070+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:20.712070+0000","last_peered":"2024-09-11T05:15:20.712070+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:20.712070+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6052,"log_dups_size":0,"ondisk_log_size":6052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4081,"num_write_kb":16324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2525}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5919","reported_seq":9780,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:20.684000+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:20.684000+0000","last_peered":"2024-09-11T05:15:20.684000+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:20.684000+0000","last_undegraded":"2024-09-11T05:15:20.684000+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1875,"num_bytes_recovered":7680000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1603},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6229","reported_seq":6356,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.046005+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:24.046005+0000","last_peered":"2024-09-11T05:15:24.046005+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:24.046005+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6229,"log_dups_size":0,"ondisk_log_size":6229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3262,"num_write_kb":13048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2604},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6058","reported_seq":6187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.073288+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:24.073288+0000","last_peered":"2024-09-11T05:15:24.073288+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:24.073288+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6058,"log_dups_size":0,"ondisk_log_size":6058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2924,"num_write_kb":11696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2555},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6064","reported_seq":6193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.016918+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:24.016918+0000","last_peered":"2024-09-11T05:15:24.016918+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:24.016918+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24821760,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2822,"num_write_kb":11288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6033","reported_seq":6172,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.041350+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:24.041350+0000","last_peered":"2024-09-11T05:15:24.041350+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:24.041350+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6033,"log_dups_size":0,"ondisk_log_size":6033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24662633,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2507},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398582377,"num_objects":97311,"num_object_clones":0,"num_object_copies":194622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1603,"num_objects_unfound":0,"num_objects_dirty":97311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60456,"num_write_kb":241824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1876,"num_bytes_recovered":7684096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802217984,"data_stored":802131009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97392,"ondisk_log_size":97392,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472516,"kb_used_data":306164,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93899324,"statfs":{"total":96636764160,"available":96152907776,"internally_reserved":0,"allocated":313511936,"data_stored":313267477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255920,"kb_used_data":125664,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94115920,"statfs":{"total":96636764160,"available":96374702080,"internally_reserved":0,"allocated":128679936,"data_stored":128354862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":133361323},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215139,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537584,"kb_used_data":349088,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93834256,"statfs":{"total":96636764160,"available":96086278144,"internally_reserved":0,"allocated":357466112,"data_stored":357217576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":192999083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358961152,"data_stored":358957673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":129376256,"data_stored":129296239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313880576,"data_stored":313877097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:27.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:27.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6217", 'reported_seq': 6291, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:24.049167+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:24.049167+0000', 'last_peered': '2024-09-11T05:15:24.049167+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:24.049167+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 6217, 'log_dups_size': 0, 'ondisk_log_size': 6217, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25464832, 'num_objects': 6217, 'num_object_clones': 0, 'num_object_copies': 12434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6217, 'num_write_kb': 24868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2542}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:27.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5964", 'reported_seq': 6084, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:20.675218+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:20.675218+0000', 'last_peered': '2024-09-11T05:15:20.675218+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:20.675218+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 5964, 'log_dups_size': 0, 'ondisk_log_size': 5964, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24412160, 'num_objects': 5960, 'num_object_clones': 0, 'num_object_copies': 11920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4367, 'num_write_kb': 17468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 2466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:27.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:27.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6134", 'reported_seq': 6278, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:23.931313+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:23.931313+0000', 'last_peered': '2024-09-11T05:15:23.931313+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:23.931313+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 6134, 'log_dups_size': 0, 'ondisk_log_size': 6134, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25104384, 'num_objects': 6129, 'num_object_clones': 0, 'num_object_copies': 12258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2678, 'num_write_kb': 10712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2529}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:27.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6211", 'reported_seq': 6339, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:23.905948+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:23.905948+0000', 'last_peered': '2024-09-11T05:15:23.905948+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:23.905948+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 6211, 'log_dups_size': 0, 'ondisk_log_size': 6211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25415680, 'num_objects': 6205, 'num_object_clones': 0, 'num_object_copies': 12410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3376, 'num_write_kb': 13504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2629}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6022", 'reported_seq': 6148, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:20.699389+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:20.699389+0000', 'last_peered': '2024-09-11T05:15:20.699389+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:20.699389+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 6022, 'log_dups_size': 0, 'ondisk_log_size': 6022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24637440, 'num_objects': 6015, 'num_object_clones': 0, 'num_object_copies': 12030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4090, 'num_write_kb': 16360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2490}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6023", 'reported_seq': 6182, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:23.991249+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:23.991249+0000', 'last_peered': '2024-09-11T05:15:23.991249+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:23.991249+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 6023, 'log_dups_size': 0, 'ondisk_log_size': 6023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24637440, 'num_objects': 6015, 'num_object_clones': 0, 'num_object_copies': 12030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2776, 'num_write_kb': 11104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2573}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6052", 'reported_seq': 6165, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:20.712070+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:20.712070+0000', 'last_peered': '2024-09-11T05:15:20.712070+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:20.712070+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 6052, 'log_dups_size': 0, 'ondisk_log_size': 6052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24768512, 'num_objects': 6047, 'num_object_clones': 0, 'num_object_copies': 12094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4081, 'num_write_kb': 16324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 2525}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:27.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5919", 'reported_seq': 9780, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:20.684000+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:20.684000+0000', 'last_peered': '2024-09-11T05:15:20.684000+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:20.684000+0000', 'last_undegraded': '2024-09-11T05:15:20.684000+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 5919, 'log_dups_size': 0, 'ondisk_log_size': 5919, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24219648, 'num_objects': 5913, 'num_object_clones': 0, 'num_object_copies': 11826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1603, 'num_objects_unfound': 0, 'num_objects_dirty': 5913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3822, 'num_write_kb': 15288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1875, 'num_bytes_recovered': 7680000, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1603}, {'shards': '2', 'objects': 4310}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:27.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:27.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6229", 'reported_seq': 6356, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:24.046005+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:24.046005+0000', 'last_peered': '2024-09-11T05:15:24.046005+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:24.046005+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 6229, 'log_dups_size': 0, 'ondisk_log_size': 6229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25505792, 'num_objects': 6227, 'num_object_clones': 0, 'num_object_copies': 12454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3262, 'num_write_kb': 13048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2604}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:27.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6058", 'reported_seq': 6187, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:24.073288+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:24.073288+0000', 'last_peered': '2024-09-11T05:15:24.073288+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:24.073288+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 6058, 'log_dups_size': 0, 'ondisk_log_size': 6058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24780800, 'num_objects': 6050, 'num_object_clones': 0, 'num_object_copies': 12100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2924, 'num_write_kb': 11696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2555}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:27.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6064", 'reported_seq': 6193, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:24.016918+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:24.016918+0000', 'last_peered': '2024-09-11T05:15:24.016918+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:24.016918+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 6064, 'log_dups_size': 0, 'ondisk_log_size': 6064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24821760, 'num_objects': 6060, 'num_object_clones': 0, 'num_object_copies': 12120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6060, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2822, 'num_write_kb': 11288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2516}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:27.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6033", 'reported_seq': 6172, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:24.041350+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:24.041350+0000', 'last_peered': '2024-09-11T05:15:24.041350+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:24.041350+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 6033, 'log_dups_size': 0, 'ondisk_log_size': 6033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24662633, 'num_objects': 6022, 'num_object_clones': 0, 'num_object_copies': 12044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3040, 'num_write_kb': 12160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2507}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:27.216 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:28.115 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 115 20 100082 100062 3.39736 1.96094 0.00812755 0.022985 2024-09-11T05:15:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:28.235 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/196586 objects degraded (19.854%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":206,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98293,"data_bytes":402596476,"bytes_used":1285091328,"bytes_avail":288625201152,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":196586,"degraded_ratio":0.19854414861689032,"misplaced_objects":1405,"misplaced_total":196586,"misplaced_ratio":0.007146999277669824,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":184955,"recovering_keys_per_sec":0,"num_objects_recovered":544,"num_bytes_recovered":2220032,"num_keys_recovered":0,"write_bytes_sec":2214684,"read_op_per_sec":0,"write_op_per_sec":540},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:15:28.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:28.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:29.039 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:29.039 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:29.095 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-11T05:15:27.398759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402596476,"num_objects":98293,"num_object_clones":0,"num_object_copies":196586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1405,"num_objects_unfound":0,"num_objects_dirty":98293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61438,"num_write_kb":245746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2077,"num_bytes_recovered":8495123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98375,"ondisk_log_size":98375,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1254972,"kb_used_data":794828,"kb_used_omap":12,"kb_used_meta":460083,"kb_avail":281860548,"statfs":{"total":289910292480,"available":288625201152,"internally_reserved":0,"allocated":813903872,"data_stored":813092761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":471124993},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,6,5,7,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26583040,"num_objects":6490,"num_object_clones":0,"num_object_copies":12980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-542,"num_objects_unfound":0,"num_objects_dirty":6490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6490,"num_write_kb":25960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":544,"num_bytes_recovered":2220032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003081"},"pg_stats":[{"pgid":"2.f","version":"43'6217","reported_seq":6291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.049167+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:24.049167+0000","last_peered":"2024-09-11T05:15:24.049167+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:24.049167+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6217,"log_dups_size":0,"ondisk_log_size":6217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25464832,"num_objects":6217,"num_object_clones":0,"num_object_copies":12434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6217,"num_write_kb":24868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2542},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6208","reported_seq":6305,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:25.684843+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:25.684843+0000","last_peered":"2024-09-11T05:15:25.684843+0000","last_clean":"2024-09-11T05:15:25.684843+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:25.684843+0000","last_undegraded":"2024-09-11T05:15:25.684843+0000","last_fullsized":"2024-09-11T05:15:25.684843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25403392,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4606,"num_write_kb":18424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6130","reported_seq":6249,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:25.694343+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:25.694343+0000","last_peered":"2024-09-11T05:15:25.694343+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:25.694343+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6130,"log_dups_size":0,"ondisk_log_size":6130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092096,"num_objects":6126,"num_object_clones":0,"num_object_copies":12252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6288","reported_seq":6354,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:25.696897+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:25.696897+0000","last_peered":"2024-09-11T05:15:25.696897+0000","last_clean":"2024-09-11T05:15:25.696897+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:25.696897+0000","last_undegraded":"2024-09-11T05:15:25.696897+0000","last_fullsized":"2024-09-11T05:15:25.696897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":6288,"num_object_clones":0,"num_object_copies":12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6215","reported_seq":6340,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:23.969724+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:23.969724+0000","last_peered":"2024-09-11T05:15:23.969724+0000","last_clean":"2024-09-11T05:15:23.969724+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:23.969724+0000","last_undegraded":"2024-09-11T05:15:23.969724+0000","last_fullsized":"2024-09-11T05:15:23.969724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3049,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6134","reported_seq":6278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.931313+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:23.931313+0000","last_peered":"2024-09-11T05:15:23.931313+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:23.931313+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6134,"log_dups_size":0,"ondisk_log_size":6134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2529},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6211","reported_seq":6339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.905948+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:23.905948+0000","last_peered":"2024-09-11T05:15:23.905948+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:23.905948+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2629},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6083","reported_seq":6202,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:24.028557+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:24.028557+0000","last_peered":"2024-09-11T05:15:24.028557+0000","last_clean":"2024-09-11T05:15:24.028557+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:24.028557+0000","last_undegraded":"2024-09-11T05:15:24.028557+0000","last_fullsized":"2024-09-11T05:15:24.028557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6083,"log_dups_size":0,"ondisk_log_size":6083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24907776,"num_objects":6081,"num_object_clones":0,"num_object_copies":12162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6200","reported_seq":6325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:25.740942+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:25.740942+0000","last_peered":"2024-09-11T05:15:25.740942+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:25.740942+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6200,"log_dups_size":0,"ondisk_log_size":6200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25366528,"num_objects":6193,"num_object_clones":0,"num_object_copies":12386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4268,"num_write_kb":17072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2668}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6023","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:23.991249+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:23.991249+0000","last_peered":"2024-09-11T05:15:23.991249+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:23.991249+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6023,"log_dups_size":0,"ondisk_log_size":6023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6219","reported_seq":6332,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:25.698695+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:25.698695+0000","last_peered":"2024-09-11T05:15:25.698695+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:25.698695+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6219,"log_dups_size":0,"ondisk_log_size":6219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25452544,"num_objects":6214,"num_object_clones":0,"num_object_copies":12428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4248,"num_write_kb":16992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6060","reported_seq":10323,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:25.730553+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:25.730553+0000","last_peered":"2024-09-11T05:15:25.730553+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:25.730553+0000","last_undegraded":"2024-09-11T05:15:25.730553+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6060,"log_dups_size":0,"ondisk_log_size":6060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24797184,"num_objects":6054,"num_object_clones":0,"num_object_copies":12108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1405,"num_objects_unfound":0,"num_objects_dirty":6054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2075,"num_bytes_recovered":8491008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1405},{"shards":"2","objects":4649}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6229","reported_seq":6356,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.046005+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:24.046005+0000","last_peered":"2024-09-11T05:15:24.046005+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:24.046005+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6229,"log_dups_size":0,"ondisk_log_size":6229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3262,"num_write_kb":13048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2604},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6058","reported_seq":6187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.073288+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:24.073288+0000","last_peered":"2024-09-11T05:15:24.073288+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:24.073288+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6058,"log_dups_size":0,"ondisk_log_size":6058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2924,"num_write_kb":11696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2555},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6064","reported_seq":6193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.016918+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:24.016918+0000","last_peered":"2024-09-11T05:15:24.016918+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:24.016918+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24821760,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2822,"num_write_kb":11288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6033","reported_seq":6172,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:24.041350+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:24.041350+0000","last_peered":"2024-09-11T05:15:24.041350+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:24.041350+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6033,"log_dups_size":0,"ondisk_log_size":6033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24662633,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2507},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":402596457,"num_objects":98291,"num_object_clones":0,"num_object_copies":196582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1405,"num_objects_unfound":0,"num_objects_dirty":98291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61436,"num_write_kb":245744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2076,"num_bytes_recovered":8495104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":816656384,"data_stored":816576367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98372,"ondisk_log_size":98372,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":311452,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":318926848,"data_stored":318682389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157544107},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":252064,"kb_used_data":134288,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94119776,"statfs":{"total":96636764160,"available":96378650624,"internally_reserved":0,"allocated":137510912,"data_stored":137192796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":120581803},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215139,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537584,"kb_used_data":349088,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93834256,"statfs":{"total":96636764160,"available":96086278144,"internally_reserved":0,"allocated":357466112,"data_stored":357217576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":192999083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358961152,"data_stored":358957673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":138448896,"data_stored":138375837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":319246336,"data_stored":319242857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:29.115 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 116 20 100597 100577 3.38541 2.01172 0.0192831 0.0230683 2024-09-11T05:15:30.115 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 117 20 101150 101130 3.37494 2.16016 0.00795603 0.0231388 2024-09-11T05:15:31.116 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 118 20 101698 101678 3.36444 2.14062 0.014429 0.0232147 2024-09-11T05:15:32.097 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:32.116 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 119 20 102247 102227 3.35419 2.14453 0.0414556 0.0232842 2024-09-11T05:15:32.884 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:32.884 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:32.963 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-11T05:15:31.399656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":413307516,"num_objects":100908,"num_object_clones":0,"num_object_copies":201816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":100908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64053,"num_write_kb":256206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2259,"num_bytes_recovered":9240595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100990,"ondisk_log_size":100990,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294844,"kb_used_data":816652,"kb_used_omap":12,"kb_used_meta":478131,"kb_avail":281820676,"statfs":{"total":289910292480,"available":288584372224,"internally_reserved":0,"allocated":836251648,"data_stored":835440537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":489606145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,4,5,7,14,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25915392,"num_objects":6327,"num_object_clones":0,"num_object_copies":12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-559,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6327,"num_write_kb":25308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":561,"num_bytes_recovered":2289664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002947"},"pg_stats":[{"pgid":"2.f","version":"43'6400","reported_seq":6473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.104578+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:29.104578+0000","last_peered":"2024-09-11T05:15:29.104578+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:29.104578+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26214400,"num_objects":6400,"num_object_clones":0,"num_object_copies":12800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6400,"num_write_kb":25600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2725},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6376","reported_seq":6471,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:30.735134+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:30.735134+0000","last_peered":"2024-09-11T05:15:30.735134+0000","last_clean":"2024-09-11T05:15:30.735134+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:30.735134+0000","last_undegraded":"2024-09-11T05:15:30.735134+0000","last_fullsized":"2024-09-11T05:15:30.735134+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6376,"log_dups_size":0,"ondisk_log_size":6376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6288","reported_seq":6408,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.740134+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:30.740134+0000","last_peered":"2024-09-11T05:15:30.740134+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:30.740134+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25739264,"num_objects":6284,"num_object_clones":0,"num_object_copies":12568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":18764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6439","reported_seq":6506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:30.676951+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:30.676951+0000","last_peered":"2024-09-11T05:15:30.676951+0000","last_clean":"2024-09-11T05:15:30.676951+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:30.676951+0000","last_undegraded":"2024-09-11T05:15:30.676951+0000","last_fullsized":"2024-09-11T05:15:30.676951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6439,"log_dups_size":0,"ondisk_log_size":6439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26374144,"num_objects":6439,"num_object_clones":0,"num_object_copies":12878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6439,"num_write_kb":25756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6376","reported_seq":6501,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:29.088433+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:29.088433+0000","last_peered":"2024-09-11T05:15:29.088433+0000","last_clean":"2024-09-11T05:15:29.088433+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:29.088433+0000","last_undegraded":"2024-09-11T05:15:29.088433+0000","last_fullsized":"2024-09-11T05:15:29.088433+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6376,"log_dups_size":0,"ondisk_log_size":6376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3210,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6309","reported_seq":6452,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.077432+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:29.077432+0000","last_peered":"2024-09-11T05:15:29.077432+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:29.077432+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6309,"log_dups_size":0,"ondisk_log_size":6309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2853,"num_write_kb":11412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2704},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6394","reported_seq":6522,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.067835+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:29.067835+0000","last_peered":"2024-09-11T05:15:29.067835+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:29.067835+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26165248,"num_objects":6388,"num_object_clones":0,"num_object_copies":12776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3559,"num_write_kb":14236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2812},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6250","reported_seq":6369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:29.075324+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:29.075324+0000","last_peered":"2024-09-11T05:15:29.075324+0000","last_clean":"2024-09-11T05:15:29.075324+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:29.075324+0000","last_undegraded":"2024-09-11T05:15:29.075324+0000","last_fullsized":"2024-09-11T05:15:29.075324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6250,"log_dups_size":0,"ondisk_log_size":6250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25591808,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6366","reported_seq":6493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.704297+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:30.704297+0000","last_peered":"2024-09-11T05:15:30.704297+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:30.704297+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6366,"log_dups_size":0,"ondisk_log_size":6366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26046464,"num_objects":6359,"num_object_clones":0,"num_object_copies":12718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4434,"num_write_kb":17736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2834}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6165","reported_seq":6325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.073246+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:29.073246+0000","last_peered":"2024-09-11T05:15:29.073246+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:29.073246+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6165,"log_dups_size":0,"ondisk_log_size":6165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2918,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6394","reported_seq":6507,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.740791+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:30.740791+0000","last_peered":"2024-09-11T05:15:30.740791+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:30.740791+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4423,"num_write_kb":17692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2867}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6205","reported_seq":10832,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:30.733619+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:30.733619+0000","last_peered":"2024-09-11T05:15:30.733619+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:30.733619+0000","last_undegraded":"2024-09-11T05:15:30.733619+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2257,"num_bytes_recovered":9236480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1223},{"shards":"2","objects":4976}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6413","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.090335+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:29.090335+0000","last_peered":"2024-09-11T05:15:29.090335+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:29.090335+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26259456,"num_objects":6411,"num_object_clones":0,"num_object_copies":12822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3446,"num_write_kb":13784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6212","reported_seq":6341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:28.999874+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:28.999874+0000","last_peered":"2024-09-11T05:15:28.999874+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:28.999874+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6212,"log_dups_size":0,"ondisk_log_size":6212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25411584,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3078,"num_write_kb":12312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6211","reported_seq":6339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:28.987515+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:28.987515+0000","last_peered":"2024-09-11T05:15:28.987515+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:28.987515+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25423872,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2969,"num_write_kb":11876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2663},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6189","reported_seq":6328,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.046051+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:29.046051+0000","last_peered":"2024-09-11T05:15:29.046051+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:29.046051+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6189,"log_dups_size":0,"ondisk_log_size":6189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25301609,"num_objects":6178,"num_object_clones":0,"num_object_copies":12356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3196,"num_write_kb":12784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2663},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":413307497,"num_objects":100906,"num_object_clones":0,"num_object_copies":201812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":100906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64051,"num_write_kb":256204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2258,"num_bytes_recovered":9240576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":838877184,"data_stored":838797167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100987,"ondisk_log_size":100987,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492712,"kb_used_data":316632,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93879128,"statfs":{"total":96636764160,"available":96132227072,"internally_reserved":0,"allocated":324231168,"data_stored":323986709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180285099},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,3,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":268864,"kb_used_data":142768,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94102976,"statfs":{"total":96636764160,"available":96361447424,"internally_reserved":0,"allocated":146194432,"data_stored":145876316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215140,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533268,"kb_used_data":357252,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93838572,"statfs":{"total":96636764160,"available":96090697728,"internally_reserved":0,"allocated":365826048,"data_stored":365577512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,1,2,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":366964736,"data_stored":366961257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147267584,"data_stored":147194525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324644864,"data_stored":324641385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:32.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:33.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:15:33.117686+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0233627 2024-09-11T05:15:33.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:15:33.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 120 20 102751 102731 3.34265 1.96875 0.0403598 0.0233627 2024-09-11T05:15:33.955 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:34.026 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-11T05:15:33.400152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":413307516,"num_objects":100908,"num_object_clones":0,"num_object_copies":201816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":100908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64053,"num_write_kb":256206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2259,"num_bytes_recovered":9240595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100990,"ondisk_log_size":100990,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294844,"kb_used_data":816652,"kb_used_omap":12,"kb_used_meta":478131,"kb_avail":281820676,"statfs":{"total":289910292480,"available":288584372224,"internally_reserved":0,"allocated":836251648,"data_stored":835440537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":489606145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,4,5,7,14,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21647360,"num_objects":5285,"num_object_clones":0,"num_object_copies":10570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":5285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5285,"num_write_kb":21140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002919"},"pg_stats":[{"pgid":"2.f","version":"43'6400","reported_seq":6473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.104578+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:29.104578+0000","last_peered":"2024-09-11T05:15:29.104578+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:29.104578+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26214400,"num_objects":6400,"num_object_clones":0,"num_object_copies":12800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6400,"num_write_kb":25600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2725},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6376","reported_seq":6471,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:30.735134+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:30.735134+0000","last_peered":"2024-09-11T05:15:30.735134+0000","last_clean":"2024-09-11T05:15:30.735134+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:30.735134+0000","last_undegraded":"2024-09-11T05:15:30.735134+0000","last_fullsized":"2024-09-11T05:15:30.735134+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6376,"log_dups_size":0,"ondisk_log_size":6376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6288","reported_seq":6408,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.740134+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:30.740134+0000","last_peered":"2024-09-11T05:15:30.740134+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:30.740134+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25739264,"num_objects":6284,"num_object_clones":0,"num_object_copies":12568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":18764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6439","reported_seq":6506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:30.676951+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:30.676951+0000","last_peered":"2024-09-11T05:15:30.676951+0000","last_clean":"2024-09-11T05:15:30.676951+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:30.676951+0000","last_undegraded":"2024-09-11T05:15:30.676951+0000","last_fullsized":"2024-09-11T05:15:30.676951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6439,"log_dups_size":0,"ondisk_log_size":6439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26374144,"num_objects":6439,"num_object_clones":0,"num_object_copies":12878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6439,"num_write_kb":25756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6376","reported_seq":6501,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:29.088433+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:29.088433+0000","last_peered":"2024-09-11T05:15:29.088433+0000","last_clean":"2024-09-11T05:15:29.088433+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:29.088433+0000","last_undegraded":"2024-09-11T05:15:29.088433+0000","last_fullsized":"2024-09-11T05:15:29.088433+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6376,"log_dups_size":0,"ondisk_log_size":6376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3210,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6309","reported_seq":6452,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.077432+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:29.077432+0000","last_peered":"2024-09-11T05:15:29.077432+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:29.077432+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6309,"log_dups_size":0,"ondisk_log_size":6309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2853,"num_write_kb":11412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2704},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6394","reported_seq":6522,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.067835+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:29.067835+0000","last_peered":"2024-09-11T05:15:29.067835+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:29.067835+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26165248,"num_objects":6388,"num_object_clones":0,"num_object_copies":12776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3559,"num_write_kb":14236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2812},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6250","reported_seq":6369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:29.075324+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:29.075324+0000","last_peered":"2024-09-11T05:15:29.075324+0000","last_clean":"2024-09-11T05:15:29.075324+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:29.075324+0000","last_undegraded":"2024-09-11T05:15:29.075324+0000","last_fullsized":"2024-09-11T05:15:29.075324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6250,"log_dups_size":0,"ondisk_log_size":6250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25591808,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6366","reported_seq":6493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.704297+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:30.704297+0000","last_peered":"2024-09-11T05:15:30.704297+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:30.704297+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6366,"log_dups_size":0,"ondisk_log_size":6366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26046464,"num_objects":6359,"num_object_clones":0,"num_object_copies":12718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4434,"num_write_kb":17736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2834}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6165","reported_seq":6325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.073246+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:29.073246+0000","last_peered":"2024-09-11T05:15:29.073246+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:29.073246+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6165,"log_dups_size":0,"ondisk_log_size":6165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2918,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6394","reported_seq":6507,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.740791+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:30.740791+0000","last_peered":"2024-09-11T05:15:30.740791+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:30.740791+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4423,"num_write_kb":17692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2867}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6205","reported_seq":10832,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:30.733619+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:30.733619+0000","last_peered":"2024-09-11T05:15:30.733619+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:30.733619+0000","last_undegraded":"2024-09-11T05:15:30.733619+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2257,"num_bytes_recovered":9236480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1223},{"shards":"2","objects":4976}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6413","reported_seq":6539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.090335+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:29.090335+0000","last_peered":"2024-09-11T05:15:29.090335+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:29.090335+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26259456,"num_objects":6411,"num_object_clones":0,"num_object_copies":12822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3446,"num_write_kb":13784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6212","reported_seq":6341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:28.999874+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:28.999874+0000","last_peered":"2024-09-11T05:15:28.999874+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:28.999874+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6212,"log_dups_size":0,"ondisk_log_size":6212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25411584,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3078,"num_write_kb":12312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6211","reported_seq":6339,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:28.987515+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:28.987515+0000","last_peered":"2024-09-11T05:15:28.987515+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:28.987515+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25423872,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2969,"num_write_kb":11876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2663},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6189","reported_seq":6328,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:29.046051+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:29.046051+0000","last_peered":"2024-09-11T05:15:29.046051+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:29.046051+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6189,"log_dups_size":0,"ondisk_log_size":6189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25301609,"num_objects":6178,"num_object_clones":0,"num_object_copies":12356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3196,"num_write_kb":12784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2663},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":413307497,"num_objects":100906,"num_object_clones":0,"num_object_copies":201812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":100906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64051,"num_write_kb":256204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2258,"num_bytes_recovered":9240576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":838877184,"data_stored":838797167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100987,"ondisk_log_size":100987,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492712,"kb_used_data":316632,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93879128,"statfs":{"total":96636764160,"available":96132227072,"internally_reserved":0,"allocated":324231168,"data_stored":323986709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180285099},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,3,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":268864,"kb_used_data":142768,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94102976,"statfs":{"total":96636764160,"available":96361447424,"internally_reserved":0,"allocated":146194432,"data_stored":145876316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215140,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533268,"kb_used_data":357252,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93838572,"statfs":{"total":96636764160,"available":96090697728,"internally_reserved":0,"allocated":365826048,"data_stored":365577512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,1,2,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":366964736,"data_stored":366961257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147267584,"data_stored":147194525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324644864,"data_stored":324641385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:34.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:34.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6400", 'reported_seq': 6473, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:29.104578+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:29.104578+0000', 'last_peered': '2024-09-11T05:15:29.104578+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:29.104578+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 6400, 'log_dups_size': 0, 'ondisk_log_size': 6400, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26214400, 'num_objects': 6400, 'num_object_clones': 0, 'num_object_copies': 12800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6400, 'num_write_kb': 25600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2725}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:34.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6288", 'reported_seq': 6408, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:30.740134+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:30.740134+0000', 'last_peered': '2024-09-11T05:15:30.740134+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:30.740134+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 6288, 'log_dups_size': 0, 'ondisk_log_size': 6288, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25739264, 'num_objects': 6284, 'num_object_clones': 0, 'num_object_copies': 12568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4691, 'num_write_kb': 18764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 2790}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:34.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:34.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6309", 'reported_seq': 6452, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:29.077432+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:29.077432+0000', 'last_peered': '2024-09-11T05:15:29.077432+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:29.077432+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 6309, 'log_dups_size': 0, 'ondisk_log_size': 6309, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25821184, 'num_objects': 6304, 'num_object_clones': 0, 'num_object_copies': 12608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6304, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2853, 'num_write_kb': 11412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2704}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6394", 'reported_seq': 6522, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:29.067835+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:29.067835+0000', 'last_peered': '2024-09-11T05:15:29.067835+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:29.067835+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 6394, 'log_dups_size': 0, 'ondisk_log_size': 6394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26165248, 'num_objects': 6388, 'num_object_clones': 0, 'num_object_copies': 12776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3559, 'num_write_kb': 14236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2812}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6366", 'reported_seq': 6493, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:30.704297+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:30.704297+0000', 'last_peered': '2024-09-11T05:15:30.704297+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:30.704297+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 6366, 'log_dups_size': 0, 'ondisk_log_size': 6366, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26046464, 'num_objects': 6359, 'num_object_clones': 0, 'num_object_copies': 12718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4434, 'num_write_kb': 17736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 2834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6165", 'reported_seq': 6325, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:29.073246+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:29.073246+0000', 'last_peered': '2024-09-11T05:15:29.073246+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:29.073246+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 6165, 'log_dups_size': 0, 'ondisk_log_size': 6165, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25219072, 'num_objects': 6157, 'num_object_clones': 0, 'num_object_copies': 12314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2918, 'num_write_kb': 11672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2715}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6394", 'reported_seq': 6507, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:30.740791+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:30.740791+0000', 'last_peered': '2024-09-11T05:15:30.740791+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:30.740791+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 6394, 'log_dups_size': 0, 'ondisk_log_size': 6394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26169344, 'num_objects': 6389, 'num_object_clones': 0, 'num_object_copies': 12778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4423, 'num_write_kb': 17692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 2867}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6205", 'reported_seq': 10832, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:30.733619+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:30.733619+0000', 'last_peered': '2024-09-11T05:15:30.733619+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:30.733619+0000', 'last_undegraded': '2024-09-11T05:15:30.733619+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 6205, 'log_dups_size': 0, 'ondisk_log_size': 6205, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25391104, 'num_objects': 6199, 'num_object_clones': 0, 'num_object_copies': 12398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1223, 'num_objects_unfound': 0, 'num_objects_dirty': 6199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4108, 'num_write_kb': 16432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2257, 'num_bytes_recovered': 9236480, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1223}, {'shards': '2', 'objects': 4976}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:34.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:34.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6413", 'reported_seq': 6539, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:29.090335+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:29.090335+0000', 'last_peered': '2024-09-11T05:15:29.090335+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:29.090335+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 6413, 'log_dups_size': 0, 'ondisk_log_size': 6413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26259456, 'num_objects': 6411, 'num_object_clones': 0, 'num_object_copies': 12822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3446, 'num_write_kb': 13784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2788}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:34.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6212", 'reported_seq': 6341, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:28.999874+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:28.999874+0000', 'last_peered': '2024-09-11T05:15:28.999874+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:28.999874+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 6212, 'log_dups_size': 0, 'ondisk_log_size': 6212, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25411584, 'num_objects': 6204, 'num_object_clones': 0, 'num_object_copies': 12408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3078, 'num_write_kb': 12312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2709}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:34.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6211", 'reported_seq': 6339, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:28.987515+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:28.987515+0000', 'last_peered': '2024-09-11T05:15:28.987515+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:28.987515+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 6211, 'log_dups_size': 0, 'ondisk_log_size': 6211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25423872, 'num_objects': 6207, 'num_object_clones': 0, 'num_object_copies': 12414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2969, 'num_write_kb': 11876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2663}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:34.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6189", 'reported_seq': 6328, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:29.046051+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:29.046051+0000', 'last_peered': '2024-09-11T05:15:29.046051+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:29.046051+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 6189, 'log_dups_size': 0, 'ondisk_log_size': 6189, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25301609, 'num_objects': 6178, 'num_object_clones': 0, 'num_object_copies': 12356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3196, 'num_write_kb': 12784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2663}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:34.030 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:34.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 121 20 103259 103239 3.33143 1.98438 0.0440446 0.0234442 2024-09-11T05:15:35.101 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:35.101 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/201816 objects degraded (19.340%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":212,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":100908,"data_bytes":413307516,"bytes_used":1325920256,"bytes_avail":288584372224,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":201816,"degraded_ratio":0.19339893764617277,"misplaced_objects":1223,"misplaced_total":201816,"misplaced_ratio":0.0060599754231577289,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":129675,"recovering_keys_per_sec":0,"num_objects_recovered":382,"num_bytes_recovered":1556480,"num_keys_recovered":0,"write_bytes_sec":1803507,"read_op_per_sec":0,"write_op_per_sec":440},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:15:35.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 122 20 103762 103742 3.32023 1.96484 0.0124679 0.02352 2024-09-11T05:15:35.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:35.197 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:35.957 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:35.958 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:36.015 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-11T05:15:35.400941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420319868,"num_objects":102620,"num_object_clones":0,"num_object_copies":205240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":102620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65765,"num_write_kb":263054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2259,"num_bytes_recovered":9240595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102702,"ondisk_log_size":102702,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1330664,"kb_used_data":826104,"kb_used_omap":12,"kb_used_meta":504499,"kb_avail":281784856,"statfs":{"total":289910292480,"available":288547692544,"internally_reserved":0,"allocated":845930496,"data_stored":845119385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":516606977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,2,5,8,13,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28659712,"num_objects":6997,"num_object_clones":0,"num_object_copies":13994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":6997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6997,"num_write_kb":27988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003183"},"pg_stats":[{"pgid":"2.f","version":"43'6562","reported_seq":6636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.109606+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:34.109606+0000","last_peered":"2024-09-11T05:15:34.109606+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:34.109606+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26877952,"num_objects":6562,"num_object_clones":0,"num_object_copies":13124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6562,"num_write_kb":26248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2887},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6376","reported_seq":6471,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:30.735134+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:30.735134+0000","last_peered":"2024-09-11T05:15:30.735134+0000","last_clean":"2024-09-11T05:15:30.735134+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:30.735134+0000","last_undegraded":"2024-09-11T05:15:30.735134+0000","last_fullsized":"2024-09-11T05:15:30.735134+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6376,"log_dups_size":0,"ondisk_log_size":6376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6288","reported_seq":6408,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.740134+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:30.740134+0000","last_peered":"2024-09-11T05:15:30.740134+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:30.740134+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25739264,"num_objects":6284,"num_object_clones":0,"num_object_copies":12568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":18764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6439","reported_seq":6506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:30.676951+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:30.676951+0000","last_peered":"2024-09-11T05:15:30.676951+0000","last_clean":"2024-09-11T05:15:30.676951+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:30.676951+0000","last_undegraded":"2024-09-11T05:15:30.676951+0000","last_fullsized":"2024-09-11T05:15:30.676951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6439,"log_dups_size":0,"ondisk_log_size":6439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26374144,"num_objects":6439,"num_object_clones":0,"num_object_copies":12878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6439,"num_write_kb":25756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6550","reported_seq":6675,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:34.113745+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:34.113745+0000","last_peered":"2024-09-11T05:15:34.113745+0000","last_clean":"2024-09-11T05:15:34.113745+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:34.113745+0000","last_undegraded":"2024-09-11T05:15:34.113745+0000","last_fullsized":"2024-09-11T05:15:34.113745+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26800128,"num_objects":6543,"num_object_clones":0,"num_object_copies":13086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3384,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6478","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.088871+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:34.088871+0000","last_peered":"2024-09-11T05:15:34.088871+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:34.088871+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3022,"num_write_kb":12088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2873},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6537","reported_seq":6665,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.051477+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:34.051477+0000","last_peered":"2024-09-11T05:15:34.051477+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:34.051477+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6537,"log_dups_size":0,"ondisk_log_size":6537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26750976,"num_objects":6531,"num_object_clones":0,"num_object_copies":13062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2955},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6430","reported_seq":6549,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:34.136483+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:34.136483+0000","last_peered":"2024-09-11T05:15:34.136483+0000","last_clean":"2024-09-11T05:15:34.136483+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:34.136483+0000","last_undegraded":"2024-09-11T05:15:34.136483+0000","last_fullsized":"2024-09-11T05:15:34.136483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6430,"log_dups_size":0,"ondisk_log_size":6430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26329088,"num_objects":6428,"num_object_clones":0,"num_object_copies":12856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3733,"num_write_kb":14932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6366","reported_seq":6493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.704297+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:30.704297+0000","last_peered":"2024-09-11T05:15:30.704297+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:30.704297+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6366,"log_dups_size":0,"ondisk_log_size":6366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26046464,"num_objects":6359,"num_object_clones":0,"num_object_copies":12718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4434,"num_write_kb":17736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":2834}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6343","reported_seq":6503,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.082672+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:34.082672+0000","last_peered":"2024-09-11T05:15:34.082672+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:34.082672+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6343,"log_dups_size":0,"ondisk_log_size":6343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25948160,"num_objects":6335,"num_object_clones":0,"num_object_copies":12670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3096,"num_write_kb":12384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2893},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6394","reported_seq":6507,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:30.740791+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:30.740791+0000","last_peered":"2024-09-11T05:15:30.740791+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:30.740791+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4423,"num_write_kb":17692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":2867}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6205","reported_seq":10832,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:30.733619+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:30.733619+0000","last_peered":"2024-09-11T05:15:30.733619+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:30.733619+0000","last_undegraded":"2024-09-11T05:15:30.733619+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2257,"num_bytes_recovered":9236480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1223},{"shards":"2","objects":4976}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6584","reported_seq":6711,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.113835+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:34.113835+0000","last_peered":"2024-09-11T05:15:34.113835+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:34.113835+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6584,"log_dups_size":0,"ondisk_log_size":6584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26959872,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3617,"num_write_kb":14468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6392","reported_seq":6519,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.035626+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:34.035626+0000","last_peered":"2024-09-11T05:15:34.035626+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:34.035626+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6392,"log_dups_size":0,"ondisk_log_size":6392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26148864,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3258,"num_write_kb":13032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2889},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6385","reported_seq":6515,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.163396+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:34.163396+0000","last_peered":"2024-09-11T05:15:34.163396+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:34.163396+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6385,"log_dups_size":0,"ondisk_log_size":6385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3143,"num_write_kb":12572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2837},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6370","reported_seq":6509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:34.133002+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:34.133002+0000","last_peered":"2024-09-11T05:15:34.133002+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:34.133002+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6370,"log_dups_size":0,"ondisk_log_size":6370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26042985,"num_objects":6359,"num_object_clones":0,"num_object_copies":12718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3377,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2844},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420319849,"num_objects":102618,"num_object_clones":0,"num_object_copies":205236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1223,"num_objects_unfound":0,"num_objects_dirty":102618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65763,"num_write_kb":263052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2258,"num_bytes_recovered":9240576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":847220736,"data_stored":847140719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102699,"ondisk_log_size":102699,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492712,"kb_used_data":316632,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93879128,"statfs":{"total":96636764160,"available":96132227072,"internally_reserved":0,"allocated":324231168,"data_stored":323986709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180285099},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,3,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":268864,"kb_used_data":142768,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94102976,"statfs":{"total":96636764160,"available":96361447424,"internally_reserved":0,"allocated":146194432,"data_stored":145876316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129101483},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215141,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569088,"kb_used_data":366704,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93802752,"statfs":{"total":96636764160,"available":96054018048,"internally_reserved":0,"allocated":375504896,"data_stored":375256360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":207220395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":375308288,"data_stored":375304809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147267584,"data_stored":147194525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324644864,"data_stored":324641385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:36.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 123 20 104277 104257 3.30959 2.01172 0.0461293 0.0235971 2024-09-11T05:15:37.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 124 20 104821 104801 3.30004 2.125 0.0236991 0.0236644 2024-09-11T05:15:38.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 125 20 105373 105353 3.29089 2.15625 0.0188388 0.0237339 2024-09-11T05:15:39.018 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:39.120 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 126 20 105907 105887 3.28128 2.08594 0.0190487 0.0238004 2024-09-11T05:15:39.890 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:39.892 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:39.967 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-11T05:15:39.402404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431116924,"num_objects":105256,"num_object_clones":0,"num_object_copies":210512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1016,"num_objects_unfound":0,"num_objects_dirty":105256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68401,"num_write_kb":273598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":10088467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105338,"ondisk_log_size":105338,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370500,"kb_used_data":847892,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281745020,"statfs":{"total":289910292480,"available":288506900480,"internally_reserved":0,"allocated":868241408,"data_stored":867430297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":535088129},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,5,10,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-389,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6963,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":389,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003644"},"pg_stats":[{"pgid":"2.f","version":"43'6732","reported_seq":6806,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.216315+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:39.216315+0000","last_peered":"2024-09-11T05:15:39.216315+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:39.216315+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6732,"log_dups_size":0,"ondisk_log_size":6732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27574272,"num_objects":6732,"num_object_clones":0,"num_object_copies":13464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6732,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3057},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6538","reported_seq":6636,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:35.735308+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:35.735308+0000","last_peered":"2024-09-11T05:15:35.735308+0000","last_clean":"2024-09-11T05:15:35.735308+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:35.735308+0000","last_undegraded":"2024-09-11T05:15:35.735308+0000","last_fullsized":"2024-09-11T05:15:35.735308+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26755072,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6424","reported_seq":6544,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:35.759832+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:35.759832+0000","last_peered":"2024-09-11T05:15:35.759832+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:35.759832+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6424,"log_dups_size":0,"ondisk_log_size":6424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4827,"num_write_kb":19308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2926}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6598","reported_seq":6665,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:35.638036+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:35.638036+0000","last_peered":"2024-09-11T05:15:35.638036+0000","last_clean":"2024-09-11T05:15:35.638036+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:35.638036+0000","last_undegraded":"2024-09-11T05:15:35.638036+0000","last_fullsized":"2024-09-11T05:15:35.638036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6598,"log_dups_size":0,"ondisk_log_size":6598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6598,"num_write_kb":26392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6719","reported_seq":6844,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:39.160341+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:39.160341+0000","last_peered":"2024-09-11T05:15:39.160341+0000","last_clean":"2024-09-11T05:15:39.160341+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:39.160341+0000","last_undegraded":"2024-09-11T05:15:39.160341+0000","last_fullsized":"2024-09-11T05:15:39.160341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27492352,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6635","reported_seq":6779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.199333+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:39.199333+0000","last_peered":"2024-09-11T05:15:39.199333+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:39.199333+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6635,"log_dups_size":0,"ondisk_log_size":6635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27156480,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3179,"num_write_kb":12716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6713","reported_seq":6841,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.176142+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:39.176142+0000","last_peered":"2024-09-11T05:15:39.176142+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:39.176142+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6713,"log_dups_size":0,"ondisk_log_size":6713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27471872,"num_objects":6707,"num_object_clones":0,"num_object_copies":13414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6578","reported_seq":6697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:39.133157+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:39.133157+0000","last_peered":"2024-09-11T05:15:39.133157+0000","last_clean":"2024-09-11T05:15:39.133157+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:39.133157+0000","last_undegraded":"2024-09-11T05:15:39.133157+0000","last_fullsized":"2024-09-11T05:15:39.133157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6578,"log_dups_size":0,"ondisk_log_size":6578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3881,"num_write_kb":15524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6534","reported_seq":6661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:35.780602+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:35.780602+0000","last_peered":"2024-09-11T05:15:35.780602+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:35.780602+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6534,"log_dups_size":0,"ondisk_log_size":6534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26734592,"num_objects":6527,"num_object_clones":0,"num_object_copies":13054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4602,"num_write_kb":18408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6525","reported_seq":6684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.200364+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:39.200364+0000","last_peered":"2024-09-11T05:15:39.200364+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:39.200364+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6525,"log_dups_size":0,"ondisk_log_size":6525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":13112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3075},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6576","reported_seq":6685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:35.777129+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:35.777129+0000","last_peered":"2024-09-11T05:15:35.777129+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:35.777129+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6576,"log_dups_size":0,"ondisk_log_size":6576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26914816,"num_objects":6571,"num_object_clones":0,"num_object_copies":13142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4605,"num_write_kb":18420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6372","reported_seq":11412,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:35.791185+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:35.791185+0000","last_peered":"2024-09-11T05:15:35.791185+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:35.791185+0000","last_undegraded":"2024-09-11T05:15:35.791185+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26075136,"num_objects":6366,"num_object_clones":0,"num_object_copies":12732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1016,"num_objects_unfound":0,"num_objects_dirty":6366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4275,"num_write_kb":17100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2464,"num_bytes_recovered":10084352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1016},{"shards":"2","objects":5350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6737","reported_seq":6864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.130794+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:39.130794+0000","last_peered":"2024-09-11T05:15:39.130794+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:39.130794+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6737,"log_dups_size":0,"ondisk_log_size":6737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27586560,"num_objects":6735,"num_object_clones":0,"num_object_copies":13470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3770,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6557","reported_seq":6686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.211302+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:39.211302+0000","last_peered":"2024-09-11T05:15:39.211302+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:39.211302+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6557,"log_dups_size":0,"ondisk_log_size":6557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3423,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3054},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6550","reported_seq":6680,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.172484+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:39.172484+0000","last_peered":"2024-09-11T05:15:39.172484+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:39.172484+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812416,"num_objects":6546,"num_object_clones":0,"num_object_copies":13092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6547","reported_seq":6686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.126148+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:39.126148+0000","last_peered":"2024-09-11T05:15:39.126148+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:39.126148+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6547,"log_dups_size":0,"ondisk_log_size":6547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26767977,"num_objects":6536,"num_object_clones":0,"num_object_copies":13072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":14216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3021},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431116905,"num_objects":105254,"num_object_clones":0,"num_object_copies":210508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1016,"num_objects_unfound":0,"num_objects_dirty":105254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68399,"num_write_kb":273596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2465,"num_bytes_recovered":10088448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":869707776,"data_stored":869627759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105335,"ondisk_log_size":105335,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506228,"kb_used_data":321828,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93865612,"statfs":{"total":96636764160,"available":96118386688,"internally_reserved":0,"allocated":329551872,"data_stored":329307413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299680,"kb_used_data":151376,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94072160,"statfs":{"total":96636764160,"available":96329891840,"internally_reserved":0,"allocated":155009024,"data_stored":154690908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151842475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215142,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564592,"kb_used_data":374688,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93807248,"statfs":{"total":96636764160,"available":96058621952,"internally_reserved":0,"allocated":383680512,"data_stored":383431976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194440875},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,0,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":383549440,"data_stored":383545961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156155904,"data_stored":156082845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330002432,"data_stored":329998953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:39.968 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:40.119 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 127 20 106401 106381 3.27064 1.92969 0.0290785 0.0238748 2024-09-11T05:15:40.889 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:40.974 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-11T05:15:39.402404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431116924,"num_objects":105256,"num_object_clones":0,"num_object_copies":210512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1016,"num_objects_unfound":0,"num_objects_dirty":105256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68401,"num_write_kb":273598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2466,"num_bytes_recovered":10088467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105338,"ondisk_log_size":105338,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370500,"kb_used_data":847892,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281745020,"statfs":{"total":289910292480,"available":288506900480,"internally_reserved":0,"allocated":868241408,"data_stored":867430297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":535088129},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,5,10,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-389,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6963,"num_write_kb":27852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":389,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003644"},"pg_stats":[{"pgid":"2.f","version":"43'6732","reported_seq":6806,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.216315+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:39.216315+0000","last_peered":"2024-09-11T05:15:39.216315+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:39.216315+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6732,"log_dups_size":0,"ondisk_log_size":6732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27574272,"num_objects":6732,"num_object_clones":0,"num_object_copies":13464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6732,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3057},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6538","reported_seq":6636,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:35.735308+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:35.735308+0000","last_peered":"2024-09-11T05:15:35.735308+0000","last_clean":"2024-09-11T05:15:35.735308+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:35.735308+0000","last_undegraded":"2024-09-11T05:15:35.735308+0000","last_fullsized":"2024-09-11T05:15:35.735308+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26755072,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6424","reported_seq":6544,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:35.759832+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:35.759832+0000","last_peered":"2024-09-11T05:15:35.759832+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:35.759832+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6424,"log_dups_size":0,"ondisk_log_size":6424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4827,"num_write_kb":19308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":2926}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6598","reported_seq":6665,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:35.638036+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:35.638036+0000","last_peered":"2024-09-11T05:15:35.638036+0000","last_clean":"2024-09-11T05:15:35.638036+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:35.638036+0000","last_undegraded":"2024-09-11T05:15:35.638036+0000","last_fullsized":"2024-09-11T05:15:35.638036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6598,"log_dups_size":0,"ondisk_log_size":6598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6598,"num_write_kb":26392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6719","reported_seq":6844,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:39.160341+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:39.160341+0000","last_peered":"2024-09-11T05:15:39.160341+0000","last_clean":"2024-09-11T05:15:39.160341+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:39.160341+0000","last_undegraded":"2024-09-11T05:15:39.160341+0000","last_fullsized":"2024-09-11T05:15:39.160341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27492352,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6635","reported_seq":6779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.199333+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:39.199333+0000","last_peered":"2024-09-11T05:15:39.199333+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:39.199333+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6635,"log_dups_size":0,"ondisk_log_size":6635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27156480,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3179,"num_write_kb":12716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6713","reported_seq":6841,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.176142+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:39.176142+0000","last_peered":"2024-09-11T05:15:39.176142+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:39.176142+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6713,"log_dups_size":0,"ondisk_log_size":6713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27471872,"num_objects":6707,"num_object_clones":0,"num_object_copies":13414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6578","reported_seq":6697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:39.133157+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:39.133157+0000","last_peered":"2024-09-11T05:15:39.133157+0000","last_clean":"2024-09-11T05:15:39.133157+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:39.133157+0000","last_undegraded":"2024-09-11T05:15:39.133157+0000","last_fullsized":"2024-09-11T05:15:39.133157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6578,"log_dups_size":0,"ondisk_log_size":6578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3881,"num_write_kb":15524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6534","reported_seq":6661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:35.780602+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:35.780602+0000","last_peered":"2024-09-11T05:15:35.780602+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:35.780602+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6534,"log_dups_size":0,"ondisk_log_size":6534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26734592,"num_objects":6527,"num_object_clones":0,"num_object_copies":13054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4602,"num_write_kb":18408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6525","reported_seq":6684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.200364+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:39.200364+0000","last_peered":"2024-09-11T05:15:39.200364+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:39.200364+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6525,"log_dups_size":0,"ondisk_log_size":6525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":13112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3075},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6576","reported_seq":6685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:35.777129+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:35.777129+0000","last_peered":"2024-09-11T05:15:35.777129+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:35.777129+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6576,"log_dups_size":0,"ondisk_log_size":6576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26914816,"num_objects":6571,"num_object_clones":0,"num_object_copies":13142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4605,"num_write_kb":18420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6372","reported_seq":11412,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:35.791185+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:35.791185+0000","last_peered":"2024-09-11T05:15:35.791185+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:35.791185+0000","last_undegraded":"2024-09-11T05:15:35.791185+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6372,"log_dups_size":0,"ondisk_log_size":6372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26075136,"num_objects":6366,"num_object_clones":0,"num_object_copies":12732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1016,"num_objects_unfound":0,"num_objects_dirty":6366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4275,"num_write_kb":17100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2464,"num_bytes_recovered":10084352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1016},{"shards":"2","objects":5350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6737","reported_seq":6864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.130794+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:39.130794+0000","last_peered":"2024-09-11T05:15:39.130794+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:39.130794+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6737,"log_dups_size":0,"ondisk_log_size":6737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27586560,"num_objects":6735,"num_object_clones":0,"num_object_copies":13470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3770,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6557","reported_seq":6686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.211302+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:39.211302+0000","last_peered":"2024-09-11T05:15:39.211302+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:39.211302+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6557,"log_dups_size":0,"ondisk_log_size":6557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3423,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3054},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6550","reported_seq":6680,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.172484+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:39.172484+0000","last_peered":"2024-09-11T05:15:39.172484+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:39.172484+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812416,"num_objects":6546,"num_object_clones":0,"num_object_copies":13092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6547","reported_seq":6686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.126148+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:39.126148+0000","last_peered":"2024-09-11T05:15:39.126148+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:39.126148+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6547,"log_dups_size":0,"ondisk_log_size":6547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26767977,"num_objects":6536,"num_object_clones":0,"num_object_copies":13072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":14216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3021},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431116905,"num_objects":105254,"num_object_clones":0,"num_object_copies":210508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":1016,"num_objects_unfound":0,"num_objects_dirty":105254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68399,"num_write_kb":273596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2465,"num_bytes_recovered":10088448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":869707776,"data_stored":869627759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105335,"ondisk_log_size":105335,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506228,"kb_used_data":321828,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93865612,"statfs":{"total":96636764160,"available":96118386688,"internally_reserved":0,"allocated":329551872,"data_stored":329307413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299680,"kb_used_data":151376,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94072160,"statfs":{"total":96636764160,"available":96329891840,"internally_reserved":0,"allocated":155009024,"data_stored":154690908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151842475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215142,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564592,"kb_used_data":374688,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93807248,"statfs":{"total":96636764160,"available":96058621952,"internally_reserved":0,"allocated":383680512,"data_stored":383431976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194440875},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,0,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":383549440,"data_stored":383545961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156155904,"data_stored":156082845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330002432,"data_stored":329998953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:40.975 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:40.975 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6732", 'reported_seq': 6806, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.216315+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:39.216315+0000', 'last_peered': '2024-09-11T05:15:39.216315+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:39.216315+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 6732, 'log_dups_size': 0, 'ondisk_log_size': 6732, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27574272, 'num_objects': 6732, 'num_object_clones': 0, 'num_object_copies': 13464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6732, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3057}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6424", 'reported_seq': 6544, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:35.759832+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:35.759832+0000', 'last_peered': '2024-09-11T05:15:35.759832+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:35.759832+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 6424, 'log_dups_size': 0, 'ondisk_log_size': 6424, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26296320, 'num_objects': 6420, 'num_object_clones': 0, 'num_object_copies': 12840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4827, 'num_write_kb': 19308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 2926}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6635", 'reported_seq': 6779, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.199333+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:39.199333+0000', 'last_peered': '2024-09-11T05:15:39.199333+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:39.199333+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 6635, 'log_dups_size': 0, 'ondisk_log_size': 6635, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27156480, 'num_objects': 6630, 'num_object_clones': 0, 'num_object_copies': 13260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3179, 'num_write_kb': 12716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3030}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6713", 'reported_seq': 6841, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.176142+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:39.176142+0000', 'last_peered': '2024-09-11T05:15:39.176142+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:39.176142+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 6713, 'log_dups_size': 0, 'ondisk_log_size': 6713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27471872, 'num_objects': 6707, 'num_object_clones': 0, 'num_object_copies': 13414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3878, 'num_write_kb': 15512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3131}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6534", 'reported_seq': 6661, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:35.780602+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:35.780602+0000', 'last_peered': '2024-09-11T05:15:35.780602+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:35.780602+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 6534, 'log_dups_size': 0, 'ondisk_log_size': 6534, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26734592, 'num_objects': 6527, 'num_object_clones': 0, 'num_object_copies': 13054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4602, 'num_write_kb': 18408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:40.976 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6525", 'reported_seq': 6684, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.200364+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:39.200364+0000', 'last_peered': '2024-09-11T05:15:39.200364+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:39.200364+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 6525, 'log_dups_size': 0, 'ondisk_log_size': 6525, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26693632, 'num_objects': 6517, 'num_object_clones': 0, 'num_object_copies': 13034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3278, 'num_write_kb': 13112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3075}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6576", 'reported_seq': 6685, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:35.777129+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:35.777129+0000', 'last_peered': '2024-09-11T05:15:35.777129+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:35.777129+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 6576, 'log_dups_size': 0, 'ondisk_log_size': 6576, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26914816, 'num_objects': 6571, 'num_object_clones': 0, 'num_object_copies': 13142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6571, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4605, 'num_write_kb': 18420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 3049}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6372", 'reported_seq': 11412, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:35.791185+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:35.791185+0000', 'last_peered': '2024-09-11T05:15:35.791185+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:35.791185+0000', 'last_undegraded': '2024-09-11T05:15:35.791185+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 6372, 'log_dups_size': 0, 'ondisk_log_size': 6372, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26075136, 'num_objects': 6366, 'num_object_clones': 0, 'num_object_copies': 12732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1016, 'num_objects_unfound': 0, 'num_objects_dirty': 6366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4275, 'num_write_kb': 17100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2464, 'num_bytes_recovered': 10084352, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1016}, {'shards': '2', 'objects': 5350}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6737", 'reported_seq': 6864, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.130794+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:39.130794+0000', 'last_peered': '2024-09-11T05:15:39.130794+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:39.130794+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 6737, 'log_dups_size': 0, 'ondisk_log_size': 6737, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27586560, 'num_objects': 6735, 'num_object_clones': 0, 'num_object_copies': 13470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6735, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3770, 'num_write_kb': 15080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3112}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6557", 'reported_seq': 6686, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.211302+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:39.211302+0000', 'last_peered': '2024-09-11T05:15:39.211302+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:39.211302+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 6557, 'log_dups_size': 0, 'ondisk_log_size': 6557, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26824704, 'num_objects': 6549, 'num_object_clones': 0, 'num_object_copies': 13098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3423, 'num_write_kb': 13692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3054}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:40.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6550", 'reported_seq': 6680, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.172484+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:39.172484+0000', 'last_peered': '2024-09-11T05:15:39.172484+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:39.172484+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 6550, 'log_dups_size': 0, 'ondisk_log_size': 6550, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26812416, 'num_objects': 6546, 'num_object_clones': 0, 'num_object_copies': 13092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3308, 'num_write_kb': 13232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3002}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:40.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6547", 'reported_seq': 6686, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:39.126148+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:39.126148+0000', 'last_peered': '2024-09-11T05:15:39.126148+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:39.126148+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 6547, 'log_dups_size': 0, 'ondisk_log_size': 6547, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26767977, 'num_objects': 6536, 'num_object_clones': 0, 'num_object_copies': 13072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6536, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3554, 'num_write_kb': 14216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3021}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:40.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:41.121 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 128 20 106905 106885 3.26047 1.96875 0.0579649 0.023953 2024-09-11T05:15:41.998 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:41.998 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/212514 objects degraded (18.366%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":219,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106257,"data_bytes":435217020,"bytes_used":1396445184,"bytes_avail":288513847296,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":212514,"degraded_ratio":0.18366319395428066,"misplaced_objects":795,"misplaced_total":212514,"misplaced_ratio":0.0037409300093170332,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":208150,"recovering_keys_per_sec":0,"num_objects_recovered":610,"num_bytes_recovered":2498560,"num_keys_recovered":0,"write_bytes_sec":2153844,"read_op_per_sec":0,"write_op_per_sec":525},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:15:42.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:42.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:42.119 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 129 20 107404 107384 3.25031 1.94922 0.0999943 0.0240278 2024-09-11T05:15:42.764 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:42.767 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:42.815 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-11T05:15:41.402902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":435217020,"num_objects":106257,"num_object_clones":0,"num_object_copies":212514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":106257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69402,"num_write_kb":277602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2687,"num_bytes_recovered":10993683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106339,"ondisk_log_size":106339,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1363716,"kb_used_data":861908,"kb_used_omap":12,"kb_used_meta":501747,"kb_avail":281751804,"statfs":{"total":289910292480,"available":288513847296,"internally_reserved":0,"allocated":882593792,"data_stored":881782681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":513788929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,9,8,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25853952,"num_objects":6312,"num_object_clones":0,"num_object_copies":12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-610,"num_objects_unfound":0,"num_objects_dirty":6312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6312,"num_write_kb":25248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":610,"num_bytes_recovered":2498560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003630"},"pg_stats":[{"pgid":"2.f","version":"43'6732","reported_seq":6806,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.216315+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:39.216315+0000","last_peered":"2024-09-11T05:15:39.216315+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:39.216315+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6732,"log_dups_size":0,"ondisk_log_size":6732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27574272,"num_objects":6732,"num_object_clones":0,"num_object_copies":13464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6732,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3057},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6720","reported_seq":6818,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:40.799890+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:40.799890+0000","last_peered":"2024-09-11T05:15:40.799890+0000","last_clean":"2024-09-11T05:15:40.799890+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:40.799890+0000","last_undegraded":"2024-09-11T05:15:40.799890+0000","last_fullsized":"2024-09-11T05:15:40.799890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6720,"log_dups_size":0,"ondisk_log_size":6720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27500544,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5118,"num_write_kb":20472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6586","reported_seq":6706,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.743993+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:40.743993+0000","last_peered":"2024-09-11T05:15:40.743993+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:40.743993+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6586,"log_dups_size":0,"ondisk_log_size":6586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26959872,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3088}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6749","reported_seq":6816,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:40.687010+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:40.687010+0000","last_peered":"2024-09-11T05:15:40.687010+0000","last_clean":"2024-09-11T05:15:40.687010+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:40.687010+0000","last_undegraded":"2024-09-11T05:15:40.687010+0000","last_fullsized":"2024-09-11T05:15:40.687010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6749,"log_dups_size":0,"ondisk_log_size":6749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27643904,"num_objects":6749,"num_object_clones":0,"num_object_copies":13498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6749,"num_write_kb":26996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6719","reported_seq":6844,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:39.160341+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:39.160341+0000","last_peered":"2024-09-11T05:15:39.160341+0000","last_clean":"2024-09-11T05:15:39.160341+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:39.160341+0000","last_undegraded":"2024-09-11T05:15:39.160341+0000","last_fullsized":"2024-09-11T05:15:39.160341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27492352,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6635","reported_seq":6779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.199333+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:39.199333+0000","last_peered":"2024-09-11T05:15:39.199333+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:39.199333+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6635,"log_dups_size":0,"ondisk_log_size":6635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27156480,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3179,"num_write_kb":12716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6713","reported_seq":6841,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.176142+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:39.176142+0000","last_peered":"2024-09-11T05:15:39.176142+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:39.176142+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6713,"log_dups_size":0,"ondisk_log_size":6713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27471872,"num_objects":6707,"num_object_clones":0,"num_object_copies":13414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":15512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6578","reported_seq":6697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:39.133157+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:39.133157+0000","last_peered":"2024-09-11T05:15:39.133157+0000","last_clean":"2024-09-11T05:15:39.133157+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:39.133157+0000","last_undegraded":"2024-09-11T05:15:39.133157+0000","last_fullsized":"2024-09-11T05:15:39.133157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6578,"log_dups_size":0,"ondisk_log_size":6578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3881,"num_write_kb":15524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6691","reported_seq":6818,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.697299+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:40.697299+0000","last_peered":"2024-09-11T05:15:40.697299+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:40.697299+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6691,"log_dups_size":0,"ondisk_log_size":6691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27377664,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4759,"num_write_kb":19036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6525","reported_seq":6684,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.200364+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:39.200364+0000","last_peered":"2024-09-11T05:15:39.200364+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:39.200364+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6525,"log_dups_size":0,"ondisk_log_size":6525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":13112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3075},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6754","reported_seq":6867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.753507+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:40.753507+0000","last_peered":"2024-09-11T05:15:40.753507+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:40.753507+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6754,"log_dups_size":0,"ondisk_log_size":6754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27643904,"num_objects":6749,"num_object_clones":0,"num_object_copies":13498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6543","reported_seq":12026,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:40.820419+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:40.820419+0000","last_peered":"2024-09-11T05:15:40.820419+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:40.820419+0000","last_undegraded":"2024-09-11T05:15:40.820419+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26775552,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4446,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2685,"num_bytes_recovered":10989568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":795},{"shards":"2","objects":5742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6737","reported_seq":6864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.130794+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:39.130794+0000","last_peered":"2024-09-11T05:15:39.130794+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:39.130794+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6737,"log_dups_size":0,"ondisk_log_size":6737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27586560,"num_objects":6735,"num_object_clones":0,"num_object_copies":13470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3770,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6557","reported_seq":6686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.211302+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:39.211302+0000","last_peered":"2024-09-11T05:15:39.211302+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:39.211302+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6557,"log_dups_size":0,"ondisk_log_size":6557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3423,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3054},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6550","reported_seq":6680,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.172484+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:39.172484+0000","last_peered":"2024-09-11T05:15:39.172484+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:39.172484+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812416,"num_objects":6546,"num_object_clones":0,"num_object_copies":13092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6547","reported_seq":6686,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:39.126148+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:39.126148+0000","last_peered":"2024-09-11T05:15:39.126148+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:39.126148+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6547,"log_dups_size":0,"ondisk_log_size":6547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26767977,"num_objects":6536,"num_object_clones":0,"num_object_copies":13072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":14216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3021},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":435217001,"num_objects":106255,"num_object_clones":0,"num_object_copies":212510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":106255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69400,"num_write_kb":277600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":10993664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":884162560,"data_stored":884082543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106336,"ondisk_log_size":106336,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498936,"kb_used_data":327016,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93872904,"statfs":{"total":96636764160,"available":96125853696,"internally_reserved":0,"allocated":334864384,"data_stored":334619925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":160204,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":164048896,"data_stored":163730780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215142,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564592,"kb_used_data":374688,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93807248,"statfs":{"total":96636764160,"available":96058621952,"internally_reserved":0,"allocated":383680512,"data_stored":383431976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194440875},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,0,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":383549440,"data_stored":383545961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165257216,"data_stored":165184157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335355904,"data_stored":335352425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:43.119 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 130 20 107920 107900 3.24081 2.01562 0.023782 0.0240973 2024-09-11T05:15:44.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 131 20 108441 108421 3.23153 2.03516 0.0257324 0.0241639 2024-09-11T05:15:45.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 132 20 108965 108945 3.22255 2.04688 0.00992956 0.0242347 2024-09-11T05:15:45.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:46.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 133 20 109473 109453 3.21325 1.98438 0.0694377 0.0243037 2024-09-11T05:15:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:46.572 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:46.611 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-11T05:15:45.403890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441713276,"num_objects":107843,"num_object_clones":0,"num_object_copies":215686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":107843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70988,"num_write_kb":283946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2687,"num_bytes_recovered":10993683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107925,"ondisk_log_size":107925,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1379540,"kb_used_data":869412,"kb_used_omap":12,"kb_used_meta":510067,"kb_avail":281735980,"statfs":{"total":289910292480,"available":288497643520,"internally_reserved":0,"allocated":890277888,"data_stored":889466777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":522308609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,2,9,6,5,3,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28405760,"num_objects":6935,"num_object_clones":0,"num_object_copies":13870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-428,"num_objects_unfound":0,"num_objects_dirty":6935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6935,"num_write_kb":27740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":428,"num_bytes_recovered":1753088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003737"},"pg_stats":[{"pgid":"2.f","version":"43'6887","reported_seq":6961,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.146627+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:44.146627+0000","last_peered":"2024-09-11T05:15:44.146627+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:44.146627+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209152,"num_objects":6887,"num_object_clones":0,"num_object_copies":13774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6887,"num_write_kb":27548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3212},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6720","reported_seq":6818,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:40.799890+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:40.799890+0000","last_peered":"2024-09-11T05:15:40.799890+0000","last_clean":"2024-09-11T05:15:40.799890+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:40.799890+0000","last_undegraded":"2024-09-11T05:15:40.799890+0000","last_fullsized":"2024-09-11T05:15:40.799890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6720,"log_dups_size":0,"ondisk_log_size":6720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27500544,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5118,"num_write_kb":20472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6586","reported_seq":6706,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.743993+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:40.743993+0000","last_peered":"2024-09-11T05:15:40.743993+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:40.743993+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6586,"log_dups_size":0,"ondisk_log_size":6586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26959872,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3088}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6749","reported_seq":6816,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:40.687010+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:40.687010+0000","last_peered":"2024-09-11T05:15:40.687010+0000","last_clean":"2024-09-11T05:15:40.687010+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:40.687010+0000","last_undegraded":"2024-09-11T05:15:40.687010+0000","last_fullsized":"2024-09-11T05:15:40.687010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6749,"log_dups_size":0,"ondisk_log_size":6749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27643904,"num_objects":6749,"num_object_clones":0,"num_object_copies":13498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6749,"num_write_kb":26996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6894","reported_seq":7019,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:44.259899+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:44.259899+0000","last_peered":"2024-09-11T05:15:44.259899+0000","last_clean":"2024-09-11T05:15:44.259899+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:44.259899+0000","last_undegraded":"2024-09-11T05:15:44.259899+0000","last_fullsized":"2024-09-11T05:15:44.259899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6894,"log_dups_size":0,"ondisk_log_size":6894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209152,"num_objects":6887,"num_object_clones":0,"num_object_copies":13774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6832","reported_seq":6976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.219468+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:44.219468+0000","last_peered":"2024-09-11T05:15:44.219468+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:44.219468+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6832,"log_dups_size":0,"ondisk_log_size":6832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963392,"num_objects":6827,"num_object_clones":0,"num_object_copies":13654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3227},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6876","reported_seq":7003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.252259+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:44.252259+0000","last_peered":"2024-09-11T05:15:44.252259+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:44.252259+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6876,"log_dups_size":0,"ondisk_log_size":6876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28139520,"num_objects":6870,"num_object_clones":0,"num_object_copies":13740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":16164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3294},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6731","reported_seq":6850,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:44.244106+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:44.244106+0000","last_peered":"2024-09-11T05:15:44.244106+0000","last_clean":"2024-09-11T05:15:44.244106+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:44.244106+0000","last_undegraded":"2024-09-11T05:15:44.244106+0000","last_fullsized":"2024-09-11T05:15:44.244106+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6731,"log_dups_size":0,"ondisk_log_size":6731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27561984,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4034,"num_write_kb":16136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6691","reported_seq":6818,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.697299+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:40.697299+0000","last_peered":"2024-09-11T05:15:40.697299+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:40.697299+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6691,"log_dups_size":0,"ondisk_log_size":6691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27377664,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4759,"num_write_kb":19036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6683","reported_seq":6843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.206002+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:44.206002+0000","last_peered":"2024-09-11T05:15:44.206002+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:44.206002+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6683,"log_dups_size":0,"ondisk_log_size":6683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27340800,"num_objects":6675,"num_object_clones":0,"num_object_copies":13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3233},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6754","reported_seq":6867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.753507+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:40.753507+0000","last_peered":"2024-09-11T05:15:40.753507+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:40.753507+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6754,"log_dups_size":0,"ondisk_log_size":6754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27643904,"num_objects":6749,"num_object_clones":0,"num_object_copies":13498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6543","reported_seq":12026,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:40.820419+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:40.820419+0000","last_peered":"2024-09-11T05:15:40.820419+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:40.820419+0000","last_undegraded":"2024-09-11T05:15:40.820419+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26775552,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4446,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2685,"num_bytes_recovered":10989568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":795},{"shards":"2","objects":5742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6887","reported_seq":7014,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.216278+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:44.216278+0000","last_peered":"2024-09-11T05:15:44.216278+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:44.216278+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3262},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6698","reported_seq":6827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.231363+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:44.231363+0000","last_peered":"2024-09-11T05:15:44.231363+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:44.231363+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6698,"log_dups_size":0,"ondisk_log_size":6698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3564,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3195},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6690","reported_seq":6818,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.198304+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:44.198304+0000","last_peered":"2024-09-11T05:15:44.198304+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:44.198304+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27385856,"num_objects":6686,"num_object_clones":0,"num_object_copies":13372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3448,"num_write_kb":13792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3142},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6701","reported_seq":6840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.216352+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:44.216352+0000","last_peered":"2024-09-11T05:15:44.216352+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:44.216352+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6701,"log_dups_size":0,"ondisk_log_size":6701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27398761,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3708,"num_write_kb":14832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3175},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441713257,"num_objects":107841,"num_object_clones":0,"num_object_copies":215682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":107841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70986,"num_write_kb":283944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":10993664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":891936768,"data_stored":891856751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107922,"ondisk_log_size":107922,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498936,"kb_used_data":327016,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93872904,"statfs":{"total":96636764160,"available":96125853696,"internally_reserved":0,"allocated":334864384,"data_stored":334619925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":160204,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":164048896,"data_stored":163730780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215143,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580416,"kb_used_data":382192,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93791424,"statfs":{"total":96636764160,"available":96042418176,"internally_reserved":0,"allocated":391364608,"data_stored":391116072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":202960555},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,1,0,2,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391323648,"data_stored":391320169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165257216,"data_stored":165184157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335355904,"data_stored":335352425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:46.612 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:47.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 134 20 109925 109905 3.20245 1.76562 0.0435737 0.0243854 2024-09-11T05:15:47.321 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:47.322 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:47.422 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-11T05:15:45.403890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441713276,"num_objects":107843,"num_object_clones":0,"num_object_copies":215686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":107843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70988,"num_write_kb":283946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2687,"num_bytes_recovered":10993683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107925,"ondisk_log_size":107925,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1379540,"kb_used_data":869412,"kb_used_omap":12,"kb_used_meta":510067,"kb_avail":281735980,"statfs":{"total":289910292480,"available":288497643520,"internally_reserved":0,"allocated":890277888,"data_stored":889466777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":522308609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,2,9,6,5,3,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28405760,"num_objects":6935,"num_object_clones":0,"num_object_copies":13870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-428,"num_objects_unfound":0,"num_objects_dirty":6935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6935,"num_write_kb":27740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":428,"num_bytes_recovered":1753088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003737"},"pg_stats":[{"pgid":"2.f","version":"43'6887","reported_seq":6961,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.146627+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:44.146627+0000","last_peered":"2024-09-11T05:15:44.146627+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:44.146627+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209152,"num_objects":6887,"num_object_clones":0,"num_object_copies":13774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6887,"num_write_kb":27548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3212},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6720","reported_seq":6818,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:40.799890+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:40.799890+0000","last_peered":"2024-09-11T05:15:40.799890+0000","last_clean":"2024-09-11T05:15:40.799890+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:40.799890+0000","last_undegraded":"2024-09-11T05:15:40.799890+0000","last_fullsized":"2024-09-11T05:15:40.799890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6720,"log_dups_size":0,"ondisk_log_size":6720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27500544,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5118,"num_write_kb":20472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6586","reported_seq":6706,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.743993+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:40.743993+0000","last_peered":"2024-09-11T05:15:40.743993+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:40.743993+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6586,"log_dups_size":0,"ondisk_log_size":6586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26959872,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3088}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6749","reported_seq":6816,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:40.687010+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:40.687010+0000","last_peered":"2024-09-11T05:15:40.687010+0000","last_clean":"2024-09-11T05:15:40.687010+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:40.687010+0000","last_undegraded":"2024-09-11T05:15:40.687010+0000","last_fullsized":"2024-09-11T05:15:40.687010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6749,"log_dups_size":0,"ondisk_log_size":6749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27643904,"num_objects":6749,"num_object_clones":0,"num_object_copies":13498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6749,"num_write_kb":26996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6894","reported_seq":7019,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:44.259899+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:44.259899+0000","last_peered":"2024-09-11T05:15:44.259899+0000","last_clean":"2024-09-11T05:15:44.259899+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:44.259899+0000","last_undegraded":"2024-09-11T05:15:44.259899+0000","last_fullsized":"2024-09-11T05:15:44.259899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":6894,"log_dups_size":0,"ondisk_log_size":6894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28209152,"num_objects":6887,"num_object_clones":0,"num_object_copies":13774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6832","reported_seq":6976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.219468+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:44.219468+0000","last_peered":"2024-09-11T05:15:44.219468+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:44.219468+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":6832,"log_dups_size":0,"ondisk_log_size":6832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963392,"num_objects":6827,"num_object_clones":0,"num_object_copies":13654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3227},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6876","reported_seq":7003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.252259+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:44.252259+0000","last_peered":"2024-09-11T05:15:44.252259+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:44.252259+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":6876,"log_dups_size":0,"ondisk_log_size":6876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28139520,"num_objects":6870,"num_object_clones":0,"num_object_copies":13740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4041,"num_write_kb":16164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3294},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6731","reported_seq":6850,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:44.244106+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:44.244106+0000","last_peered":"2024-09-11T05:15:44.244106+0000","last_clean":"2024-09-11T05:15:44.244106+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:44.244106+0000","last_undegraded":"2024-09-11T05:15:44.244106+0000","last_fullsized":"2024-09-11T05:15:44.244106+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6731,"log_dups_size":0,"ondisk_log_size":6731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27561984,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4034,"num_write_kb":16136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6691","reported_seq":6818,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.697299+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:40.697299+0000","last_peered":"2024-09-11T05:15:40.697299+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:40.697299+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6691,"log_dups_size":0,"ondisk_log_size":6691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27377664,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4759,"num_write_kb":19036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6683","reported_seq":6843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.206002+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:44.206002+0000","last_peered":"2024-09-11T05:15:44.206002+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:44.206002+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6683,"log_dups_size":0,"ondisk_log_size":6683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27340800,"num_objects":6675,"num_object_clones":0,"num_object_copies":13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3233},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6754","reported_seq":6867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:40.753507+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:40.753507+0000","last_peered":"2024-09-11T05:15:40.753507+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:40.753507+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6754,"log_dups_size":0,"ondisk_log_size":6754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27643904,"num_objects":6749,"num_object_clones":0,"num_object_copies":13498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6543","reported_seq":12026,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:40.820419+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:40.820419+0000","last_peered":"2024-09-11T05:15:40.820419+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:40.820419+0000","last_undegraded":"2024-09-11T05:15:40.820419+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26775552,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4446,"num_write_kb":17784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2685,"num_bytes_recovered":10989568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":795},{"shards":"2","objects":5742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6887","reported_seq":7014,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.216278+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:44.216278+0000","last_peered":"2024-09-11T05:15:44.216278+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:44.216278+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3262},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6698","reported_seq":6827,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.231363+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:44.231363+0000","last_peered":"2024-09-11T05:15:44.231363+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:44.231363+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6698,"log_dups_size":0,"ondisk_log_size":6698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3564,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3195},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6690","reported_seq":6818,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.198304+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:44.198304+0000","last_peered":"2024-09-11T05:15:44.198304+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:44.198304+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6690,"log_dups_size":0,"ondisk_log_size":6690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27385856,"num_objects":6686,"num_object_clones":0,"num_object_copies":13372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3448,"num_write_kb":13792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3142},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6701","reported_seq":6840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:44.216352+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:44.216352+0000","last_peered":"2024-09-11T05:15:44.216352+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:44.216352+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6701,"log_dups_size":0,"ondisk_log_size":6701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27398761,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3708,"num_write_kb":14832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3175},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441713257,"num_objects":107841,"num_object_clones":0,"num_object_copies":215682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":795,"num_objects_unfound":0,"num_objects_dirty":107841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70986,"num_write_kb":283944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":10993664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":891936768,"data_stored":891856751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107922,"ondisk_log_size":107922,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498936,"kb_used_data":327016,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93872904,"statfs":{"total":96636764160,"available":96125853696,"internally_reserved":0,"allocated":334864384,"data_stored":334619925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":160204,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":164048896,"data_stored":163730780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143322795},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215143,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580416,"kb_used_data":382192,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93791424,"statfs":{"total":96636764160,"available":96042418176,"internally_reserved":0,"allocated":391364608,"data_stored":391116072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":202960555},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,1,0,2,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391323648,"data_stored":391320169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165257216,"data_stored":165184157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335355904,"data_stored":335352425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:47.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:47.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6887", 'reported_seq': 6961, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.146627+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:44.146627+0000', 'last_peered': '2024-09-11T05:15:44.146627+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:44.146627+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 6887, 'log_dups_size': 0, 'ondisk_log_size': 6887, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28209152, 'num_objects': 6887, 'num_object_clones': 0, 'num_object_copies': 13774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6887, 'num_write_kb': 27548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3212}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:47.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6586", 'reported_seq': 6706, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:40.743993+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:40.743993+0000', 'last_peered': '2024-09-11T05:15:40.743993+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:40.743993+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 6586, 'log_dups_size': 0, 'ondisk_log_size': 6586, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26959872, 'num_objects': 6582, 'num_object_clones': 0, 'num_object_copies': 13164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4989, 'num_write_kb': 19956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 3088}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6832", 'reported_seq': 6976, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.219468+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:44.219468+0000', 'last_peered': '2024-09-11T05:15:44.219468+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:44.219468+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 6832, 'log_dups_size': 0, 'ondisk_log_size': 6832, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27963392, 'num_objects': 6827, 'num_object_clones': 0, 'num_object_copies': 13654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3376, 'num_write_kb': 13504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3227}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6876", 'reported_seq': 7003, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.252259+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:44.252259+0000', 'last_peered': '2024-09-11T05:15:44.252259+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:44.252259+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 6876, 'log_dups_size': 0, 'ondisk_log_size': 6876, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28139520, 'num_objects': 6870, 'num_object_clones': 0, 'num_object_copies': 13740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4041, 'num_write_kb': 16164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3294}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6691", 'reported_seq': 6818, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:40.697299+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:40.697299+0000', 'last_peered': '2024-09-11T05:15:40.697299+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:40.697299+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 6691, 'log_dups_size': 0, 'ondisk_log_size': 6691, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27377664, 'num_objects': 6684, 'num_object_clones': 0, 'num_object_copies': 13368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6684, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4759, 'num_write_kb': 19036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6683", 'reported_seq': 6843, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.206002+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:44.206002+0000', 'last_peered': '2024-09-11T05:15:44.206002+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:44.206002+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 6683, 'log_dups_size': 0, 'ondisk_log_size': 6683, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27340800, 'num_objects': 6675, 'num_object_clones': 0, 'num_object_copies': 13350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3436, 'num_write_kb': 13744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3233}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:47.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6754", 'reported_seq': 6867, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:40.753507+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:40.753507+0000', 'last_peered': '2024-09-11T05:15:40.753507+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:40.753507+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 6754, 'log_dups_size': 0, 'ondisk_log_size': 6754, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27643904, 'num_objects': 6749, 'num_object_clones': 0, 'num_object_copies': 13498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4783, 'num_write_kb': 19132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 3227}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6543", 'reported_seq': 12026, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:40.820419+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:40.820419+0000', 'last_peered': '2024-09-11T05:15:40.820419+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:40.820419+0000', 'last_undegraded': '2024-09-11T05:15:40.820419+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 6543, 'log_dups_size': 0, 'ondisk_log_size': 6543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26775552, 'num_objects': 6537, 'num_object_clones': 0, 'num_object_copies': 13074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 795, 'num_objects_unfound': 0, 'num_objects_dirty': 6537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4446, 'num_write_kb': 17784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2685, 'num_bytes_recovered': 10989568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 795}, {'shards': '2', 'objects': 5742}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6887", 'reported_seq': 7014, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.216278+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:44.216278+0000', 'last_peered': '2024-09-11T05:15:44.216278+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:44.216278+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 6887, 'log_dups_size': 0, 'ondisk_log_size': 6887, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28200960, 'num_objects': 6885, 'num_object_clones': 0, 'num_object_copies': 13770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3920, 'num_write_kb': 15680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3262}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6698", 'reported_seq': 6827, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.231363+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:44.231363+0000', 'last_peered': '2024-09-11T05:15:44.231363+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:44.231363+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 6698, 'log_dups_size': 0, 'ondisk_log_size': 6698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27402240, 'num_objects': 6690, 'num_object_clones': 0, 'num_object_copies': 13380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3564, 'num_write_kb': 14256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3195}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6690", 'reported_seq': 6818, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.198304+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:44.198304+0000', 'last_peered': '2024-09-11T05:15:44.198304+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:44.198304+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 6690, 'log_dups_size': 0, 'ondisk_log_size': 6690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27385856, 'num_objects': 6686, 'num_object_clones': 0, 'num_object_copies': 13372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3448, 'num_write_kb': 13792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3142}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:47.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6701", 'reported_seq': 6840, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:44.216352+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:44.216352+0000', 'last_peered': '2024-09-11T05:15:44.216352+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:44.216352+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 6701, 'log_dups_size': 0, 'ondisk_log_size': 6701, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27398761, 'num_objects': 6690, 'num_object_clones': 0, 'num_object_copies': 13380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3708, 'num_write_kb': 14832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3175}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:47.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:48.127 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 135 20 110426 110406 3.19313 1.95703 0.054529 0.0244588 2024-09-11T05:15:48.610 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:48.610 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/217640 objects degraded (17.934%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":226,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":108820,"data_bytes":445715068,"bytes_used":1459724288,"bytes_avail":288450568192,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":217640,"degraded_ratio":0.17933743797096122,"misplaced_objects":550,"misplaced_total":217640,"misplaced_ratio":0.0025271089873185075,"recovering_objects_per_sec":56,"recovering_bytes_per_sec":229651,"recovering_keys_per_sec":0,"num_objects_recovered":673,"num_bytes_recovered":2756608,"num_keys_recovered":0,"write_bytes_sec":2115656,"read_op_per_sec":0,"write_op_per_sec":516},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:15:48.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:48.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:49.127 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 136 20 110936 110916 3.18431 1.99219 0.0219823 0.0245255 2024-09-11T05:15:49.661 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:49.664 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:49.720 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-11T05:15:49.408954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452276860,"num_objects":110422,"num_object_clones":0,"num_object_copies":220844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":550,"num_objects_unfound":0,"num_objects_dirty":110422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73567,"num_write_kb":294262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2932,"num_bytes_recovered":11997203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110504,"ondisk_log_size":110504,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491300,"kb_used_data":892340,"kb_used_omap":12,"kb_used_meta":598899,"kb_avail":281624220,"statfs":{"total":289910292480,"available":288383201280,"internally_reserved":0,"allocated":913756160,"data_stored":912945049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":613272577},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,4,12,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-466,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6828,"num_write_kb":27312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":1908736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007541"},"pg_stats":[{"pgid":"2.f","version":"43'7045","reported_seq":7119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.227482+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:49.227482+0000","last_peered":"2024-09-11T05:15:49.227482+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:49.227482+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7045,"log_dups_size":0,"ondisk_log_size":7045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3370},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6887","reported_seq":6985,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:45.737992+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:45.737992+0000","last_peered":"2024-09-11T05:15:45.737992+0000","last_clean":"2024-09-11T05:15:45.737992+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:45.737992+0000","last_undegraded":"2024-09-11T05:15:45.737992+0000","last_fullsized":"2024-09-11T05:15:45.737992+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5285,"num_write_kb":21140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6765","reported_seq":6885,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:45.814488+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:45.814488+0000","last_peered":"2024-09-11T05:15:45.814488+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:45.814488+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6765,"log_dups_size":0,"ondisk_log_size":6765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693056,"num_objects":6761,"num_object_clones":0,"num_object_copies":13522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5168,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6894","reported_seq":6961,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:45.826176+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:45.826176+0000","last_peered":"2024-09-11T05:15:45.826176+0000","last_clean":"2024-09-11T05:15:45.826176+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:45.826176+0000","last_undegraded":"2024-09-11T05:15:45.826176+0000","last_fullsized":"2024-09-11T05:15:45.826176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":6894,"log_dups_size":0,"ondisk_log_size":6894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28237824,"num_objects":6894,"num_object_clones":0,"num_object_copies":13788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7050","reported_seq":7175,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:49.239030+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:49.239030+0000","last_peered":"2024-09-11T05:15:49.239030+0000","last_clean":"2024-09-11T05:15:49.239030+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:49.239030+0000","last_undegraded":"2024-09-11T05:15:49.239030+0000","last_fullsized":"2024-09-11T05:15:49.239030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7050,"log_dups_size":0,"ondisk_log_size":7050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28848128,"num_objects":7043,"num_object_clones":0,"num_object_copies":14086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3884,"num_write_kb":15536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7009","reported_seq":7151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.285599+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:49.285599+0000","last_peered":"2024-09-11T05:15:49.285599+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:49.285599+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7009,"log_dups_size":0,"ondisk_log_size":7009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3404},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7044","reported_seq":7169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.288450+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:49.288450+0000","last_peered":"2024-09-11T05:15:49.288450+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:49.288450+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4209,"num_write_kb":16836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3462},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6875","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:49.264537+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:49.264537+0000","last_peered":"2024-09-11T05:15:49.264537+0000","last_clean":"2024-09-11T05:15:49.264537+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:49.264537+0000","last_undegraded":"2024-09-11T05:15:49.264537+0000","last_fullsized":"2024-09-11T05:15:49.264537+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4178,"num_write_kb":16712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6877","reported_seq":7002,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:45.858020+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:45.858020+0000","last_peered":"2024-09-11T05:15:45.858020+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:45.858020+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28139520,"num_objects":6870,"num_object_clones":0,"num_object_copies":13740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4945,"num_write_kb":19780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6814","reported_seq":6974,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.231958+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:49.231958+0000","last_peered":"2024-09-11T05:15:49.231958+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:49.231958+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27877376,"num_objects":6806,"num_object_clones":0,"num_object_copies":13612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3567,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6897","reported_seq":7010,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:45.707543+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:45.707543+0000","last_peered":"2024-09-11T05:15:45.707543+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:45.707543+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":6897,"log_dups_size":0,"ondisk_log_size":6897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28229632,"num_objects":6892,"num_object_clones":0,"num_object_copies":13784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4926,"num_write_kb":19704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3370}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6700","reported_seq":12672,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:45.855089+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:45.855089+0000","last_peered":"2024-09-11T05:15:45.855089+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:45.855089+0000","last_undegraded":"2024-09-11T05:15:45.855089+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6700,"log_dups_size":0,"ondisk_log_size":6700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":550,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4603,"num_write_kb":18412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2930,"num_bytes_recovered":11993088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":550},{"shards":"2","objects":6144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7058","reported_seq":7184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.257988+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:49.257988+0000","last_peered":"2024-09-11T05:15:49.257988+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:49.257988+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7058,"log_dups_size":0,"ondisk_log_size":7058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901376,"num_objects":7056,"num_object_clones":0,"num_object_copies":14112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4091,"num_write_kb":16364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3433},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6859","reported_seq":6988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.145816+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:49.145816+0000","last_peered":"2024-09-11T05:15:49.145816+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:49.145816+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6859,"log_dups_size":0,"ondisk_log_size":6859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28061696,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3725,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3356},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6873","reported_seq":7003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.235725+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:49.235725+0000","last_peered":"2024-09-11T05:15:49.235725+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:49.235725+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3631,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3325},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6854","reported_seq":6989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.254649+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:49.254649+0000","last_peered":"2024-09-11T05:15:49.254649+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:49.254649+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6854,"log_dups_size":0,"ondisk_log_size":6854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28025449,"num_objects":6843,"num_object_clones":0,"num_object_copies":13686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3861,"num_write_kb":15444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3328},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452276841,"num_objects":110420,"num_object_clones":0,"num_object_copies":220840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":550,"num_objects_unfound":0,"num_objects_dirty":110420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73565,"num_write_kb":294260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2931,"num_bytes_recovered":11997184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":914010112,"data_stored":913930095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110501,"ondisk_log_size":110501,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508228,"kb_used_data":332148,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93863612,"statfs":{"total":96636764160,"available":96116338688,"internally_reserved":0,"allocated":340119552,"data_stored":339875093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180285099},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":336868,"kb_used_data":170516,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94034972,"statfs":{"total":96636764160,"available":96291811328,"internally_reserved":0,"allocated":174608384,"data_stored":174290268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215144,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646204,"kb_used_data":389676,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93725636,"statfs":{"total":96636764160,"available":95975051264,"internally_reserved":0,"allocated":399028224,"data_stored":398779688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262663851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":399060992,"data_stored":399057513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174264320,"data_stored":174191261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340684800,"data_stored":340681321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:50.127 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 137 20 111432 111412 3.17521 1.9375 0.0613723 0.0245949 2024-09-11T05:15:51.127 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 138 20 111950 111930 3.16686 2.02344 0.00934703 0.0246607 2024-09-11T05:15:52.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 139 20 112489 112469 3.1592 2.10547 0.0152422 0.0247198 2024-09-11T05:15:52.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:53.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:15:53.130028+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.024785 2024-09-11T05:15:53.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:15:53.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 140 20 113007 112987 3.15108 2.02344 0.0218377 0.024785 2024-09-11T05:15:53.684 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:53.687 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:53.767 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-11T05:15:53.410804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":455971452,"num_objects":111324,"num_object_clones":0,"num_object_copies":222648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":111324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74469,"num_write_kb":297870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3187,"num_bytes_recovered":13041683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111406,"ondisk_log_size":111406,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500312,"kb_used_data":905512,"kb_used_omap":12,"kb_used_meta":594739,"kb_avail":281615208,"statfs":{"total":289910292480,"available":288373972992,"internally_reserved":0,"allocated":927244288,"data_stored":926433177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":609012737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,3,7,11,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20754432,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-500,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5067,"num_write_kb":20268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":500,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007901"},"pg_stats":[{"pgid":"2.f","version":"43'7045","reported_seq":7119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.227482+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:49.227482+0000","last_peered":"2024-09-11T05:15:49.227482+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:49.227482+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7045,"log_dups_size":0,"ondisk_log_size":7045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3370},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7044","reported_seq":7142,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:50.795719+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:50.795719+0000","last_peered":"2024-09-11T05:15:50.795719+0000","last_clean":"2024-09-11T05:15:50.795719+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:50.795719+0000","last_undegraded":"2024-09-11T05:15:50.795719+0000","last_fullsized":"2024-09-11T05:15:50.795719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5442,"num_write_kb":21768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6924","reported_seq":7044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.866493+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:50.866493+0000","last_peered":"2024-09-11T05:15:50.866493+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:50.866493+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6924,"log_dups_size":0,"ondisk_log_size":6924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5327,"num_write_kb":21308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7030","reported_seq":7097,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:50.754356+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:50.754356+0000","last_peered":"2024-09-11T05:15:50.754356+0000","last_clean":"2024-09-11T05:15:50.754356+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:50.754356+0000","last_undegraded":"2024-09-11T05:15:50.754356+0000","last_fullsized":"2024-09-11T05:15:50.754356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7030,"num_write_kb":28120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7050","reported_seq":7175,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:49.239030+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:49.239030+0000","last_peered":"2024-09-11T05:15:49.239030+0000","last_clean":"2024-09-11T05:15:49.239030+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:49.239030+0000","last_undegraded":"2024-09-11T05:15:49.239030+0000","last_fullsized":"2024-09-11T05:15:49.239030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7050,"log_dups_size":0,"ondisk_log_size":7050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28848128,"num_objects":7043,"num_object_clones":0,"num_object_copies":14086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3884,"num_write_kb":15536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7009","reported_seq":7151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.285599+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:49.285599+0000","last_peered":"2024-09-11T05:15:49.285599+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:49.285599+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7009,"log_dups_size":0,"ondisk_log_size":7009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3404},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7044","reported_seq":7169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.288450+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:49.288450+0000","last_peered":"2024-09-11T05:15:49.288450+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:49.288450+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4209,"num_write_kb":16836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3462},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6875","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:49.264537+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:49.264537+0000","last_peered":"2024-09-11T05:15:49.264537+0000","last_clean":"2024-09-11T05:15:49.264537+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:49.264537+0000","last_undegraded":"2024-09-11T05:15:49.264537+0000","last_fullsized":"2024-09-11T05:15:49.264537+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4178,"num_write_kb":16712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7028","reported_seq":7155,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.847526+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:50.847526+0000","last_peered":"2024-09-11T05:15:50.847526+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:50.847526+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7028,"log_dups_size":0,"ondisk_log_size":7028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758016,"num_objects":7021,"num_object_clones":0,"num_object_copies":14042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5096,"num_write_kb":20384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6814","reported_seq":6974,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.231958+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:49.231958+0000","last_peered":"2024-09-11T05:15:49.231958+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:49.231958+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27877376,"num_objects":6806,"num_object_clones":0,"num_object_copies":13612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3567,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7031","reported_seq":7144,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.783549+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:50.783549+0000","last_peered":"2024-09-11T05:15:50.783549+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:50.783549+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7031,"log_dups_size":0,"ondisk_log_size":7031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28778496,"num_objects":7026,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5060,"num_write_kb":20240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6865","reported_seq":13349,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:50.864753+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:50.864753+0000","last_peered":"2024-09-11T05:15:50.864753+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:50.864753+0000","last_undegraded":"2024-09-11T05:15:50.864753+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6865,"log_dups_size":0,"ondisk_log_size":6865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4768,"num_write_kb":19072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3185,"num_bytes_recovered":13037568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":295},{"shards":"2","objects":6564}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7058","reported_seq":7184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.257988+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:49.257988+0000","last_peered":"2024-09-11T05:15:49.257988+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:49.257988+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7058,"log_dups_size":0,"ondisk_log_size":7058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901376,"num_objects":7056,"num_object_clones":0,"num_object_copies":14112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4091,"num_write_kb":16364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3433},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6859","reported_seq":6988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.145816+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:49.145816+0000","last_peered":"2024-09-11T05:15:49.145816+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:49.145816+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6859,"log_dups_size":0,"ondisk_log_size":6859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28061696,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3725,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3356},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6873","reported_seq":7003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.235725+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:49.235725+0000","last_peered":"2024-09-11T05:15:49.235725+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:49.235725+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3631,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3325},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6854","reported_seq":6989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.254649+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:49.254649+0000","last_peered":"2024-09-11T05:15:49.254649+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:49.254649+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6854,"log_dups_size":0,"ondisk_log_size":6854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28025449,"num_objects":6843,"num_object_clones":0,"num_object_copies":13686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3861,"num_write_kb":15444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3328},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":455971433,"num_objects":111322,"num_object_clones":0,"num_object_copies":222644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":111322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74467,"num_write_kb":297868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3186,"num_bytes_recovered":13041664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":927764480,"data_stored":927684463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111403,"ondisk_log_size":111403,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517020,"kb_used_data":336780,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93854820,"statfs":{"total":96636764160,"available":96107335680,"internally_reserved":0,"allocated":344862720,"data_stored":344618261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184544939},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337088,"kb_used_data":179056,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94034752,"statfs":{"total":96636764160,"available":96291586048,"internally_reserved":0,"allocated":183353344,"data_stored":183035228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215144,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646204,"kb_used_data":389676,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93725636,"statfs":{"total":96636764160,"available":95975051264,"internally_reserved":0,"allocated":399028224,"data_stored":398779688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262663851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":399060992,"data_stored":399057513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183132160,"data_stored":183059101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":345571328,"data_stored":345567849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:53.768 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:54.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 141 20 113506 113486 3.14256 1.94922 0.0492829 0.0248517 2024-09-11T05:15:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-11T05:15:53.410804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":455971452,"num_objects":111324,"num_object_clones":0,"num_object_copies":222648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":111324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74469,"num_write_kb":297870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3187,"num_bytes_recovered":13041683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111406,"ondisk_log_size":111406,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500312,"kb_used_data":905512,"kb_used_omap":12,"kb_used_meta":594739,"kb_avail":281615208,"statfs":{"total":289910292480,"available":288373972992,"internally_reserved":0,"allocated":927244288,"data_stored":926433177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":609012737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,3,7,11,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20754432,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-500,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5067,"num_write_kb":20268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":500,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007901"},"pg_stats":[{"pgid":"2.f","version":"43'7045","reported_seq":7119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.227482+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:49.227482+0000","last_peered":"2024-09-11T05:15:49.227482+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:49.227482+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7045,"log_dups_size":0,"ondisk_log_size":7045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":28180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3370},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7044","reported_seq":7142,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:50.795719+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:50.795719+0000","last_peered":"2024-09-11T05:15:50.795719+0000","last_clean":"2024-09-11T05:15:50.795719+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:50.795719+0000","last_undegraded":"2024-09-11T05:15:50.795719+0000","last_fullsized":"2024-09-11T05:15:50.795719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5442,"num_write_kb":21768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6924","reported_seq":7044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.866493+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:50.866493+0000","last_peered":"2024-09-11T05:15:50.866493+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:50.866493+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6924,"log_dups_size":0,"ondisk_log_size":6924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5327,"num_write_kb":21308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7030","reported_seq":7097,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:50.754356+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:50.754356+0000","last_peered":"2024-09-11T05:15:50.754356+0000","last_clean":"2024-09-11T05:15:50.754356+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:50.754356+0000","last_undegraded":"2024-09-11T05:15:50.754356+0000","last_fullsized":"2024-09-11T05:15:50.754356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7030,"num_write_kb":28120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7050","reported_seq":7175,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:49.239030+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:49.239030+0000","last_peered":"2024-09-11T05:15:49.239030+0000","last_clean":"2024-09-11T05:15:49.239030+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:49.239030+0000","last_undegraded":"2024-09-11T05:15:49.239030+0000","last_fullsized":"2024-09-11T05:15:49.239030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7050,"log_dups_size":0,"ondisk_log_size":7050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28848128,"num_objects":7043,"num_object_clones":0,"num_object_copies":14086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3884,"num_write_kb":15536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7009","reported_seq":7151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.285599+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:49.285599+0000","last_peered":"2024-09-11T05:15:49.285599+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:49.285599+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7009,"log_dups_size":0,"ondisk_log_size":7009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28688384,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3553,"num_write_kb":14212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3404},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7044","reported_seq":7169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.288450+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:49.288450+0000","last_peered":"2024-09-11T05:15:49.288450+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:49.288450+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4209,"num_write_kb":16836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3462},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6875","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:49.264537+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:49.264537+0000","last_peered":"2024-09-11T05:15:49.264537+0000","last_clean":"2024-09-11T05:15:49.264537+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:49.264537+0000","last_undegraded":"2024-09-11T05:15:49.264537+0000","last_fullsized":"2024-09-11T05:15:49.264537+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":6875,"log_dups_size":0,"ondisk_log_size":6875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4178,"num_write_kb":16712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7028","reported_seq":7155,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.847526+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:50.847526+0000","last_peered":"2024-09-11T05:15:50.847526+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:50.847526+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7028,"log_dups_size":0,"ondisk_log_size":7028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758016,"num_objects":7021,"num_object_clones":0,"num_object_copies":14042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5096,"num_write_kb":20384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6814","reported_seq":6974,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.231958+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:49.231958+0000","last_peered":"2024-09-11T05:15:49.231958+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:49.231958+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27877376,"num_objects":6806,"num_object_clones":0,"num_object_copies":13612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3567,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7031","reported_seq":7144,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.783549+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:50.783549+0000","last_peered":"2024-09-11T05:15:50.783549+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:50.783549+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7031,"log_dups_size":0,"ondisk_log_size":7031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28778496,"num_objects":7026,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5060,"num_write_kb":20240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6865","reported_seq":13349,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:50.864753+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:50.864753+0000","last_peered":"2024-09-11T05:15:50.864753+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:50.864753+0000","last_undegraded":"2024-09-11T05:15:50.864753+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6865,"log_dups_size":0,"ondisk_log_size":6865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4768,"num_write_kb":19072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3185,"num_bytes_recovered":13037568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":295},{"shards":"2","objects":6564}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7058","reported_seq":7184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.257988+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:49.257988+0000","last_peered":"2024-09-11T05:15:49.257988+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:49.257988+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7058,"log_dups_size":0,"ondisk_log_size":7058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901376,"num_objects":7056,"num_object_clones":0,"num_object_copies":14112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4091,"num_write_kb":16364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3433},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6859","reported_seq":6988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.145816+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:49.145816+0000","last_peered":"2024-09-11T05:15:49.145816+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:49.145816+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6859,"log_dups_size":0,"ondisk_log_size":6859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28061696,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3725,"num_write_kb":14900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3356},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6873","reported_seq":7003,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.235725+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:49.235725+0000","last_peered":"2024-09-11T05:15:49.235725+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:49.235725+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3631,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3325},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6854","reported_seq":6989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:49.254649+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:49.254649+0000","last_peered":"2024-09-11T05:15:49.254649+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:49.254649+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":6854,"log_dups_size":0,"ondisk_log_size":6854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28025449,"num_objects":6843,"num_object_clones":0,"num_object_copies":13686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3861,"num_write_kb":15444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3328},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":455971433,"num_objects":111322,"num_object_clones":0,"num_object_copies":222644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":111322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74467,"num_write_kb":297868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3186,"num_bytes_recovered":13041664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":927764480,"data_stored":927684463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111403,"ondisk_log_size":111403,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517020,"kb_used_data":336780,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93854820,"statfs":{"total":96636764160,"available":96107335680,"internally_reserved":0,"allocated":344862720,"data_stored":344618261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184544939},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337088,"kb_used_data":179056,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94034752,"statfs":{"total":96636764160,"available":96291586048,"internally_reserved":0,"allocated":183353344,"data_stored":183035228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215144,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646204,"kb_used_data":389676,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93725636,"statfs":{"total":96636764160,"available":95975051264,"internally_reserved":0,"allocated":399028224,"data_stored":398779688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262663851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":399060992,"data_stored":399057513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183132160,"data_stored":183059101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":345571328,"data_stored":345567849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7045", 'reported_seq': 7119, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.227482+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:49.227482+0000', 'last_peered': '2024-09-11T05:15:49.227482+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:49.227482+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 7045, 'log_dups_size': 0, 'ondisk_log_size': 7045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28856320, 'num_objects': 7045, 'num_object_clones': 0, 'num_object_copies': 14090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7045, 'num_write_kb': 28180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3370}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6924", 'reported_seq': 7044, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:50.866493+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:50.866493+0000', 'last_peered': '2024-09-11T05:15:50.866493+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:50.866493+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 6924, 'log_dups_size': 0, 'ondisk_log_size': 6924, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28344320, 'num_objects': 6920, 'num_object_clones': 0, 'num_object_copies': 13840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5327, 'num_write_kb': 21308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 3426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7009", 'reported_seq': 7151, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.285599+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:49.285599+0000', 'last_peered': '2024-09-11T05:15:49.285599+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:49.285599+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 7009, 'log_dups_size': 0, 'ondisk_log_size': 7009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28688384, 'num_objects': 7004, 'num_object_clones': 0, 'num_object_copies': 14008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3553, 'num_write_kb': 14212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3404}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7044", 'reported_seq': 7169, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.288450+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:49.288450+0000', 'last_peered': '2024-09-11T05:15:49.288450+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:49.288450+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 7044, 'log_dups_size': 0, 'ondisk_log_size': 7044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28827648, 'num_objects': 7038, 'num_object_clones': 0, 'num_object_copies': 14076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4209, 'num_write_kb': 16836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3462}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:15:54.699 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7028", 'reported_seq': 7155, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:50.847526+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:50.847526+0000', 'last_peered': '2024-09-11T05:15:50.847526+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:50.847526+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 7028, 'log_dups_size': 0, 'ondisk_log_size': 7028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28758016, 'num_objects': 7021, 'num_object_clones': 0, 'num_object_copies': 14042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5096, 'num_write_kb': 20384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3496}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6814", 'reported_seq': 6974, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.231958+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:49.231958+0000', 'last_peered': '2024-09-11T05:15:49.231958+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:49.231958+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 6814, 'log_dups_size': 0, 'ondisk_log_size': 6814, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27877376, 'num_objects': 6806, 'num_object_clones': 0, 'num_object_copies': 13612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6806, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3567, 'num_write_kb': 14268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3364}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7031", 'reported_seq': 7144, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:50.783549+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:50.783549+0000', 'last_peered': '2024-09-11T05:15:50.783549+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:50.783549+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 7031, 'log_dups_size': 0, 'ondisk_log_size': 7031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28778496, 'num_objects': 7026, 'num_object_clones': 0, 'num_object_copies': 14052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5060, 'num_write_kb': 20240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 3504}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6865", 'reported_seq': 13349, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:50.864753+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:50.864753+0000', 'last_peered': '2024-09-11T05:15:50.864753+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:50.864753+0000', 'last_undegraded': '2024-09-11T05:15:50.864753+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 6865, 'log_dups_size': 0, 'ondisk_log_size': 6865, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28094464, 'num_objects': 6859, 'num_object_clones': 0, 'num_object_copies': 13718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 295, 'num_objects_unfound': 0, 'num_objects_dirty': 6859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4768, 'num_write_kb': 19072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3185, 'num_bytes_recovered': 13037568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 295}, {'shards': '2', 'objects': 6564}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7058", 'reported_seq': 7184, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.257988+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:49.257988+0000', 'last_peered': '2024-09-11T05:15:49.257988+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:49.257988+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 7058, 'log_dups_size': 0, 'ondisk_log_size': 7058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28901376, 'num_objects': 7056, 'num_object_clones': 0, 'num_object_copies': 14112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4091, 'num_write_kb': 16364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3433}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:15:54.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6859", 'reported_seq': 6988, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.145816+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:49.145816+0000', 'last_peered': '2024-09-11T05:15:49.145816+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:49.145816+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 6859, 'log_dups_size': 0, 'ondisk_log_size': 6859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28061696, 'num_objects': 6851, 'num_object_clones': 0, 'num_object_copies': 13702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3725, 'num_write_kb': 14900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3356}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:15:54.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6873", 'reported_seq': 7003, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.235725+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:49.235725+0000', 'last_peered': '2024-09-11T05:15:49.235725+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:49.235725+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 6873, 'log_dups_size': 0, 'ondisk_log_size': 6873, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28135424, 'num_objects': 6869, 'num_object_clones': 0, 'num_object_copies': 13738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3631, 'num_write_kb': 14524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3325}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:15:54.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6854", 'reported_seq': 6989, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:49.254649+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:49.254649+0000', 'last_peered': '2024-09-11T05:15:49.254649+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:49.254649+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 6854, 'log_dups_size': 0, 'ondisk_log_size': 6854, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28025449, 'num_objects': 6843, 'num_object_clones': 0, 'num_object_copies': 13686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3861, 'num_write_kb': 15444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3328}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:15:54.701 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:15:55.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 142 20 113988 113968 3.13369 1.88281 0.0185839 0.0249236 2024-09-11T05:15:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:55.527 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39031/225740 objects degraded (17.290%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":233,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112870,"data_bytes":462303868,"bytes_used":1554137088,"bytes_avail":288356155392,"bytes_total":289910292480,"degraded_objects":39031,"degraded_total":225740,"degraded_ratio":0.17290245415079294,"misplaced_objects":295,"misplaced_total":225740,"misplaced_ratio":0.0013068131478692303,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":170554,"recovering_keys_per_sec":0,"num_objects_recovered":500,"num_bytes_recovered":2048000,"num_keys_recovered":0,"write_bytes_sec":2255752,"read_op_per_sec":0,"write_op_per_sec":550},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:15:55.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:15:55.589 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:15:56.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 143 20 114471 114451 3.12491 1.88672 0.00786087 0.0249921 2024-09-11T05:15:56.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:15:56.520 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:15:56.588 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-11T05:15:55.411302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":462303868,"num_objects":112870,"num_object_clones":0,"num_object_copies":225740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":112870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76015,"num_write_kb":304054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3187,"num_bytes_recovered":13041683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112952,"ondisk_log_size":112952,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517712,"kb_used_data":913184,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":281597808,"statfs":{"total":289910292480,"available":288356155392,"internally_reserved":0,"allocated":935100416,"data_stored":934289305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":618974209},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,2,7,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27086848,"num_objects":6613,"num_object_clones":0,"num_object_copies":13226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-500,"num_objects_unfound":0,"num_objects_dirty":6613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":500,"num_bytes_recovered":2048000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007898"},"pg_stats":[{"pgid":"2.f","version":"43'7207","reported_seq":7281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.340081+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:54.340081+0000","last_peered":"2024-09-11T05:15:54.340081+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:54.340081+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7207,"log_dups_size":0,"ondisk_log_size":7207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7207,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7044","reported_seq":7142,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:50.795719+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:50.795719+0000","last_peered":"2024-09-11T05:15:50.795719+0000","last_clean":"2024-09-11T05:15:50.795719+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:50.795719+0000","last_undegraded":"2024-09-11T05:15:50.795719+0000","last_fullsized":"2024-09-11T05:15:50.795719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5442,"num_write_kb":21768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6924","reported_seq":7044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.866493+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:50.866493+0000","last_peered":"2024-09-11T05:15:50.866493+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:50.866493+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":6924,"log_dups_size":0,"ondisk_log_size":6924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5327,"num_write_kb":21308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7030","reported_seq":7097,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:50.754356+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:50.754356+0000","last_peered":"2024-09-11T05:15:50.754356+0000","last_clean":"2024-09-11T05:15:50.754356+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:50.754356+0000","last_undegraded":"2024-09-11T05:15:50.754356+0000","last_fullsized":"2024-09-11T05:15:50.754356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7030,"num_write_kb":28120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7205","reported_seq":7330,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:54.294179+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:54.294179+0000","last_peered":"2024-09-11T05:15:54.294179+0000","last_clean":"2024-09-11T05:15:54.294179+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:54.294179+0000","last_undegraded":"2024-09-11T05:15:54.294179+0000","last_fullsized":"2024-09-11T05:15:54.294179+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483008,"num_objects":7198,"num_object_clones":0,"num_object_copies":14396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4039,"num_write_kb":16156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7184","reported_seq":7326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.281620+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:54.281620+0000","last_peered":"2024-09-11T05:15:54.281620+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:54.281620+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3579},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7206","reported_seq":7334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.320415+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:54.320415+0000","last_peered":"2024-09-11T05:15:54.320415+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:54.320415+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7206,"log_dups_size":0,"ondisk_log_size":7206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":7200,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":17484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7020","reported_seq":7139,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:54.268286+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:54.268286+0000","last_peered":"2024-09-11T05:15:54.268286+0000","last_clean":"2024-09-11T05:15:54.268286+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:54.268286+0000","last_undegraded":"2024-09-11T05:15:54.268286+0000","last_fullsized":"2024-09-11T05:15:54.268286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7020,"log_dups_size":0,"ondisk_log_size":7020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28745728,"num_objects":7018,"num_object_clones":0,"num_object_copies":14036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4323,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7028","reported_seq":7155,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.847526+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:50.847526+0000","last_peered":"2024-09-11T05:15:50.847526+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:50.847526+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7028,"log_dups_size":0,"ondisk_log_size":7028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758016,"num_objects":7021,"num_object_clones":0,"num_object_copies":14042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5096,"num_write_kb":20384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6949","reported_seq":7109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.176515+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:54.176515+0000","last_peered":"2024-09-11T05:15:54.176515+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:54.176515+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6949,"log_dups_size":0,"ondisk_log_size":6949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28430336,"num_objects":6941,"num_object_clones":0,"num_object_copies":13882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3499},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7031","reported_seq":7144,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:50.783549+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:50.783549+0000","last_peered":"2024-09-11T05:15:50.783549+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:50.783549+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7031,"log_dups_size":0,"ondisk_log_size":7031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28778496,"num_objects":7026,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5060,"num_write_kb":20240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6865","reported_seq":13349,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:50.864753+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:50.864753+0000","last_peered":"2024-09-11T05:15:50.864753+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:50.864753+0000","last_undegraded":"2024-09-11T05:15:50.864753+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":6865,"log_dups_size":0,"ondisk_log_size":6865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4768,"num_write_kb":19072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3185,"num_bytes_recovered":13037568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":295},{"shards":"2","objects":6564}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7221","reported_seq":7346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.326195+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:54.326195+0000","last_peered":"2024-09-11T05:15:54.326195+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:54.326195+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7221,"log_dups_size":0,"ondisk_log_size":7221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29569024,"num_objects":7219,"num_object_clones":0,"num_object_copies":14438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":17016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3596},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6990","reported_seq":7119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.257848+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:54.257848+0000","last_peered":"2024-09-11T05:15:54.257848+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:54.257848+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6990,"log_dups_size":0,"ondisk_log_size":6990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28598272,"num_objects":6982,"num_object_clones":0,"num_object_copies":13964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3856,"num_write_kb":15424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3487},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7030","reported_seq":7160,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.311101+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:54.311101+0000","last_peered":"2024-09-11T05:15:54.311101+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:54.311101+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28778496,"num_objects":7026,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3788,"num_write_kb":15152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3482},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7015","reported_seq":7154,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.251096+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:54.251096+0000","last_peered":"2024-09-11T05:15:54.251096+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:54.251096+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684905,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":16088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3489},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":462303849,"num_objects":112868,"num_object_clones":0,"num_object_copies":225736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":295,"num_objects_unfound":0,"num_objects_dirty":112868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76013,"num_write_kb":304052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3186,"num_bytes_recovered":13041664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":935555072,"data_stored":935475055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112949,"ondisk_log_size":112949,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517020,"kb_used_data":336780,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93854820,"statfs":{"total":96636764160,"available":96107335680,"internally_reserved":0,"allocated":344862720,"data_stored":344618261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184544939},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337088,"kb_used_data":179056,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94034752,"statfs":{"total":96636764160,"available":96291586048,"internally_reserved":0,"allocated":183353344,"data_stored":183035228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215145,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663604,"kb_used_data":397348,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93708236,"statfs":{"total":96636764160,"available":95957233664,"internally_reserved":0,"allocated":406884352,"data_stored":406635816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":406851584,"data_stored":406848105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183132160,"data_stored":183059101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":345571328,"data_stored":345567849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:15:57.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 144 20 114950 114930 3.1162 1.87109 0.0046546 0.0250272 2024-09-11T05:15:58.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 145 20 115267 115247 3.10326 1.23828 0.0282271 0.0251681 2024-09-11T05:15:59.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 146 20 115793 115773 3.09608 2.05469 0.0455831 0.0252251 2024-09-11T05:15:59.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:00.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 147 20 116309 116289 3.08873 2.01562 0.0366528 0.0252834 2024-09-11T05:16:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:00.428 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:00.502 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-11T05:15:59.412081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":466449020,"num_objects":113882,"num_object_clones":0,"num_object_copies":227764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":113882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77027,"num_write_kb":308102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3455,"num_bytes_recovered":14139411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113964,"ondisk_log_size":113964,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540704,"kb_used_data":927856,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281574816,"statfs":{"total":289910292480,"available":288332611584,"internally_reserved":0,"allocated":950124544,"data_stored":949313433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":627493889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,5,6,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10477568,"num_objects":2558,"num_object_clones":0,"num_object_copies":5116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-268,"num_objects_unfound":0,"num_objects_dirty":2558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":1097728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001770"},"pg_stats":[{"pgid":"2.f","version":"43'7207","reported_seq":7281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.340081+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:54.340081+0000","last_peered":"2024-09-11T05:15:54.340081+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:54.340081+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7207,"log_dups_size":0,"ondisk_log_size":7207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7207,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7224","reported_seq":7322,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:55.824651+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:55.824651+0000","last_peered":"2024-09-11T05:15:55.824651+0000","last_clean":"2024-09-11T05:15:55.824651+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:55.824651+0000","last_undegraded":"2024-09-11T05:15:55.824651+0000","last_fullsized":"2024-09-11T05:15:55.824651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7224,"log_dups_size":0,"ondisk_log_size":7224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29564928,"num_objects":7218,"num_object_clones":0,"num_object_copies":14436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5622,"num_write_kb":22488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7072","reported_seq":7192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:55.668384+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:55.668384+0000","last_peered":"2024-09-11T05:15:55.668384+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:55.668384+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7072,"log_dups_size":0,"ondisk_log_size":7072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28950528,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5475,"num_write_kb":21900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7202","reported_seq":7269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:55.832351+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:55.832351+0000","last_peered":"2024-09-11T05:15:55.832351+0000","last_clean":"2024-09-11T05:15:55.832351+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:55.832351+0000","last_undegraded":"2024-09-11T05:15:55.832351+0000","last_fullsized":"2024-09-11T05:15:55.832351+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29499392,"num_objects":7202,"num_object_clones":0,"num_object_copies":14404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7202,"num_write_kb":28808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7205","reported_seq":7330,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:54.294179+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:54.294179+0000","last_peered":"2024-09-11T05:15:54.294179+0000","last_clean":"2024-09-11T05:15:54.294179+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:54.294179+0000","last_undegraded":"2024-09-11T05:15:54.294179+0000","last_fullsized":"2024-09-11T05:15:54.294179+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483008,"num_objects":7198,"num_object_clones":0,"num_object_copies":14396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4039,"num_write_kb":16156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7184","reported_seq":7326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.281620+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:54.281620+0000","last_peered":"2024-09-11T05:15:54.281620+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:54.281620+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3579},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7206","reported_seq":7334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.320415+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:54.320415+0000","last_peered":"2024-09-11T05:15:54.320415+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:54.320415+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7206,"log_dups_size":0,"ondisk_log_size":7206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":7200,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":17484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7020","reported_seq":7139,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:54.268286+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:54.268286+0000","last_peered":"2024-09-11T05:15:54.268286+0000","last_clean":"2024-09-11T05:15:54.268286+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:54.268286+0000","last_undegraded":"2024-09-11T05:15:54.268286+0000","last_fullsized":"2024-09-11T05:15:54.268286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7020,"log_dups_size":0,"ondisk_log_size":7020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28745728,"num_objects":7018,"num_object_clones":0,"num_object_copies":14036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4323,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7208","reported_seq":7335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:55.872542+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:55.872542+0000","last_peered":"2024-09-11T05:15:55.872542+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:55.872542+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7208,"log_dups_size":0,"ondisk_log_size":7208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29495296,"num_objects":7201,"num_object_clones":0,"num_object_copies":14402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5276,"num_write_kb":21104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6949","reported_seq":7109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.176515+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:54.176515+0000","last_peered":"2024-09-11T05:15:54.176515+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:54.176515+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6949,"log_dups_size":0,"ondisk_log_size":6949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28430336,"num_objects":6941,"num_object_clones":0,"num_object_copies":13882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3499},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7185","reported_seq":7297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:55.854594+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:55.854594+0000","last_peered":"2024-09-11T05:15:55.854594+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:55.854594+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7185,"log_dups_size":0,"ondisk_log_size":7185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29409280,"num_objects":7180,"num_object_clones":0,"num_object_copies":14360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5214,"num_write_kb":20856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7043","reported_seq":14062,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:55.865316+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:55.865316+0000","last_peered":"2024-09-11T05:15:55.865316+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:55.865316+0000","last_undegraded":"2024-09-11T05:15:55.865316+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7043,"log_dups_size":0,"ondisk_log_size":7043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":19784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3453,"num_bytes_recovered":14135296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":27},{"shards":"2","objects":7010}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7221","reported_seq":7346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.326195+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:54.326195+0000","last_peered":"2024-09-11T05:15:54.326195+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:54.326195+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7221,"log_dups_size":0,"ondisk_log_size":7221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29569024,"num_objects":7219,"num_object_clones":0,"num_object_copies":14438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":17016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3596},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6990","reported_seq":7119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.257848+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:54.257848+0000","last_peered":"2024-09-11T05:15:54.257848+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:54.257848+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6990,"log_dups_size":0,"ondisk_log_size":6990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28598272,"num_objects":6982,"num_object_clones":0,"num_object_copies":13964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3856,"num_write_kb":15424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3487},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7030","reported_seq":7160,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.311101+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:54.311101+0000","last_peered":"2024-09-11T05:15:54.311101+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:54.311101+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28778496,"num_objects":7026,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3788,"num_write_kb":15152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3482},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7015","reported_seq":7154,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.251096+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:54.251096+0000","last_peered":"2024-09-11T05:15:54.251096+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:54.251096+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684905,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":16088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3489},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":466449001,"num_objects":113880,"num_object_clones":0,"num_object_copies":227760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":113880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77025,"num_write_kb":308100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3454,"num_bytes_recovered":14139392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949764096,"data_stored":949684079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113961,"ondisk_log_size":113961,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527436,"kb_used_data":343036,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93844404,"statfs":{"total":96636764160,"available":96096669696,"internally_reserved":0,"allocated":351268864,"data_stored":351024405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349664,"kb_used_data":187472,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94022176,"statfs":{"total":96636764160,"available":96278708224,"internally_reserved":0,"allocated":191971328,"data_stored":191653212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":166063787},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215145,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663604,"kb_used_data":397348,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93708236,"statfs":{"total":96636764160,"available":95957233664,"internally_reserved":0,"allocated":406884352,"data_stored":406635816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":406851584,"data_stored":406848105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":191930368,"data_stored":191857309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350982144,"data_stored":350978665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:00.503 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:01.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 148 20 116808 116788 3.08103 1.94922 0.0497353 0.025348 2024-09-11T05:16:01.333 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:01.392 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-11T05:15:59.412081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":466449020,"num_objects":113882,"num_object_clones":0,"num_object_copies":227764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":113882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77027,"num_write_kb":308102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3455,"num_bytes_recovered":14139411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113964,"ondisk_log_size":113964,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540704,"kb_used_data":927856,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281574816,"statfs":{"total":289910292480,"available":288332611584,"internally_reserved":0,"allocated":950124544,"data_stored":949313433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":627493889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,5,6,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10477568,"num_objects":2558,"num_object_clones":0,"num_object_copies":5116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-268,"num_objects_unfound":0,"num_objects_dirty":2558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":1097728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001770"},"pg_stats":[{"pgid":"2.f","version":"43'7207","reported_seq":7281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.340081+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:54.340081+0000","last_peered":"2024-09-11T05:15:54.340081+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:54.340081+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7207,"log_dups_size":0,"ondisk_log_size":7207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7207,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7224","reported_seq":7322,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:55.824651+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:15:55.824651+0000","last_peered":"2024-09-11T05:15:55.824651+0000","last_clean":"2024-09-11T05:15:55.824651+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:15:55.824651+0000","last_undegraded":"2024-09-11T05:15:55.824651+0000","last_fullsized":"2024-09-11T05:15:55.824651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7224,"log_dups_size":0,"ondisk_log_size":7224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29564928,"num_objects":7218,"num_object_clones":0,"num_object_copies":14436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5622,"num_write_kb":22488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7072","reported_seq":7192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:55.668384+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:15:55.668384+0000","last_peered":"2024-09-11T05:15:55.668384+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:15:55.668384+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7072,"log_dups_size":0,"ondisk_log_size":7072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28950528,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5475,"num_write_kb":21900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7202","reported_seq":7269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:55.832351+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:15:55.832351+0000","last_peered":"2024-09-11T05:15:55.832351+0000","last_clean":"2024-09-11T05:15:55.832351+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:15:55.832351+0000","last_undegraded":"2024-09-11T05:15:55.832351+0000","last_fullsized":"2024-09-11T05:15:55.832351+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29499392,"num_objects":7202,"num_object_clones":0,"num_object_copies":14404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7202,"num_write_kb":28808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7205","reported_seq":7330,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:54.294179+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:54.294179+0000","last_peered":"2024-09-11T05:15:54.294179+0000","last_clean":"2024-09-11T05:15:54.294179+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:54.294179+0000","last_undegraded":"2024-09-11T05:15:54.294179+0000","last_fullsized":"2024-09-11T05:15:54.294179+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29483008,"num_objects":7198,"num_object_clones":0,"num_object_copies":14396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4039,"num_write_kb":16156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7184","reported_seq":7326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.281620+0000","last_change":"2024-09-11T05:14:14.201598+0000","last_active":"2024-09-11T05:15:54.281620+0000","last_peered":"2024-09-11T05:15:54.281620+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:54.281620+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3579},{"shards":"0,2","objects":3598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7206","reported_seq":7334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.320415+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:54.320415+0000","last_peered":"2024-09-11T05:15:54.320415+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:54.320415+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7206,"log_dups_size":0,"ondisk_log_size":7206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":7200,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":17484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7020","reported_seq":7139,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:15:54.268286+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:54.268286+0000","last_peered":"2024-09-11T05:15:54.268286+0000","last_clean":"2024-09-11T05:15:54.268286+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:54.268286+0000","last_undegraded":"2024-09-11T05:15:54.268286+0000","last_fullsized":"2024-09-11T05:15:54.268286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7020,"log_dups_size":0,"ondisk_log_size":7020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28745728,"num_objects":7018,"num_object_clones":0,"num_object_copies":14036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4323,"num_write_kb":17292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7208","reported_seq":7335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:55.872542+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:15:55.872542+0000","last_peered":"2024-09-11T05:15:55.872542+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:15:55.872542+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7208,"log_dups_size":0,"ondisk_log_size":7208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29495296,"num_objects":7201,"num_object_clones":0,"num_object_copies":14402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5276,"num_write_kb":21104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.741271+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:14:13.741271+0000","last_peered":"2024-09-11T05:14:13.741271+0000","last_clean":"2024-09-11T05:14:13.741271+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:14:13.741271+0000","last_undegraded":"2024-09-11T05:14:13.741271+0000","last_fullsized":"2024-09-11T05:14:13.741271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6949","reported_seq":7109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.176515+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:54.176515+0000","last_peered":"2024-09-11T05:15:54.176515+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:54.176515+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":6949,"log_dups_size":0,"ondisk_log_size":6949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28430336,"num_objects":6941,"num_object_clones":0,"num_object_copies":13882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3499},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.906255+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:14:13.906255+0000","last_peered":"2024-09-11T05:14:13.906255+0000","last_clean":"2024-09-11T05:14:13.906255+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:14:13.906255+0000","last_undegraded":"2024-09-11T05:14:13.906255+0000","last_fullsized":"2024-09-11T05:14:13.906255+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7185","reported_seq":7297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:55.854594+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:15:55.854594+0000","last_peered":"2024-09-11T05:15:55.854594+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:15:55.854594+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7185,"log_dups_size":0,"ondisk_log_size":7185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29409280,"num_objects":7180,"num_object_clones":0,"num_object_copies":14360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5214,"num_write_kb":20856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7043","reported_seq":14062,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T05:15:55.865316+0000","last_change":"2024-09-11T05:14:14.182448+0000","last_active":"2024-09-11T05:15:55.865316+0000","last_peered":"2024-09-11T05:15:55.865316+0000","last_clean":"2024-09-11T05:14:12.117949+0000","last_became_active":"2024-09-11T05:14:14.147388+0000","last_became_peered":"2024-09-11T05:14:14.147388+0000","last_unstale":"2024-09-11T05:15:55.865316+0000","last_undegraded":"2024-09-11T05:15:55.865316+0000","last_fullsized":"2024-09-11T05:14:13.736687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7043,"log_dups_size":0,"ondisk_log_size":7043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":19784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3453,"num_bytes_recovered":14135296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":27},{"shards":"2","objects":7010}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7221","reported_seq":7346,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.326195+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:54.326195+0000","last_peered":"2024-09-11T05:15:54.326195+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:54.326195+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7221,"log_dups_size":0,"ondisk_log_size":7221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29569024,"num_objects":7219,"num_object_clones":0,"num_object_copies":14438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":17016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3596},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6990","reported_seq":7119,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.257848+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:54.257848+0000","last_peered":"2024-09-11T05:15:54.257848+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:54.257848+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":6990,"log_dups_size":0,"ondisk_log_size":6990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28598272,"num_objects":6982,"num_object_clones":0,"num_object_copies":13964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3856,"num_write_kb":15424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3487},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.738991+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:14:13.738991+0000","last_peered":"2024-09-11T05:14:13.738991+0000","last_clean":"2024-09-11T05:14:13.738991+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:14:13.738991+0000","last_undegraded":"2024-09-11T05:14:13.738991+0000","last_fullsized":"2024-09-11T05:14:13.738991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7030","reported_seq":7160,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.311101+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:54.311101+0000","last_peered":"2024-09-11T05:15:54.311101+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:54.311101+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28778496,"num_objects":7026,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3788,"num_write_kb":15152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3482},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7015","reported_seq":7154,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:54.251096+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:54.251096+0000","last_peered":"2024-09-11T05:15:54.251096+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:54.251096+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684905,"num_objects":7004,"num_object_clones":0,"num_object_copies":14008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4022,"num_write_kb":16088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3489},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":466449001,"num_objects":113880,"num_object_clones":0,"num_object_copies":227760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39031,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":113880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77025,"num_write_kb":308100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3454,"num_bytes_recovered":14139392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949764096,"data_stored":949684079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113961,"ondisk_log_size":113961,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527436,"kb_used_data":343036,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93844404,"statfs":{"total":96636764160,"available":96096669696,"internally_reserved":0,"allocated":351268864,"data_stored":351024405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349664,"kb_used_data":187472,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94022176,"statfs":{"total":96636764160,"available":96278708224,"internally_reserved":0,"allocated":191971328,"data_stored":191653212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":166063787},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215145,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663604,"kb_used_data":397348,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93708236,"statfs":{"total":96636764160,"available":95957233664,"internally_reserved":0,"allocated":406884352,"data_stored":406635816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":406851584,"data_stored":406848105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":191930368,"data_stored":191857309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350982144,"data_stored":350978665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:01.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:01.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7207", 'reported_seq': 7281, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.340081+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:15:54.340081+0000', 'last_peered': '2024-09-11T05:15:54.340081+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:15:54.340081+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 7207, 'log_dups_size': 0, 'ondisk_log_size': 7207, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29519872, 'num_objects': 7207, 'num_object_clones': 0, 'num_object_copies': 14414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7207, 'num_write_kb': 28828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3532}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7072", 'reported_seq': 7192, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:55.668384+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:15:55.668384+0000', 'last_peered': '2024-09-11T05:15:55.668384+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:15:55.668384+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 7072, 'log_dups_size': 0, 'ondisk_log_size': 7072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28950528, 'num_objects': 7068, 'num_object_clones': 0, 'num_object_copies': 14136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5475, 'num_write_kb': 21900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 3574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7184", 'reported_seq': 7326, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.281620+0000', 'last_change': '2024-09-11T05:14:14.201598+0000', 'last_active': '2024-09-11T05:15:54.281620+0000', 'last_peered': '2024-09-11T05:15:54.281620+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:15:54.281620+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 7184, 'log_dups_size': 0, 'ondisk_log_size': 7184, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29405184, 'num_objects': 7179, 'num_object_clones': 0, 'num_object_copies': 14358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3602, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7179, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3728, 'num_write_kb': 14912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3579}, {'shards': '0,2', 'objects': 3598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7206", 'reported_seq': 7334, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.320415+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:15:54.320415+0000', 'last_peered': '2024-09-11T05:15:54.320415+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:15:54.320415+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 7206, 'log_dups_size': 0, 'ondisk_log_size': 7206, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29491200, 'num_objects': 7200, 'num_object_clones': 0, 'num_object_copies': 14400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4371, 'num_write_kb': 17484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3624}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7208", 'reported_seq': 7335, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:55.872542+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:15:55.872542+0000', 'last_peered': '2024-09-11T05:15:55.872542+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:15:55.872542+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 7208, 'log_dups_size': 0, 'ondisk_log_size': 7208, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29495296, 'num_objects': 7201, 'num_object_clones': 0, 'num_object_copies': 14402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5276, 'num_write_kb': 21104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6949", 'reported_seq': 7109, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.176515+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:15:54.176515+0000', 'last_peered': '2024-09-11T05:15:54.176515+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:15:54.176515+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 6949, 'log_dups_size': 0, 'ondisk_log_size': 6949, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28430336, 'num_objects': 6941, 'num_object_clones': 0, 'num_object_copies': 13882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3702, 'num_write_kb': 14808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3499}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7185", 'reported_seq': 7297, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:55.854594+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:15:55.854594+0000', 'last_peered': '2024-09-11T05:15:55.854594+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:15:55.854594+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 7185, 'log_dups_size': 0, 'ondisk_log_size': 7185, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29409280, 'num_objects': 7180, 'num_object_clones': 0, 'num_object_copies': 14360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5214, 'num_write_kb': 20856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 3658}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7043", 'reported_seq': 14062, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T05:15:55.865316+0000', 'last_change': '2024-09-11T05:14:14.182448+0000', 'last_active': '2024-09-11T05:15:55.865316+0000', 'last_peered': '2024-09-11T05:15:55.865316+0000', 'last_clean': '2024-09-11T05:14:12.117949+0000', 'last_became_active': '2024-09-11T05:14:14.147388+0000', 'last_became_peered': '2024-09-11T05:14:14.147388+0000', 'last_unstale': '2024-09-11T05:15:55.865316+0000', 'last_undegraded': '2024-09-11T05:15:55.865316+0000', 'last_fullsized': '2024-09-11T05:14:13.736687+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2129", 'last_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:51.225394+0000', 'objects_scrubbed': 2091, 'log_size': 7043, 'log_dups_size': 0, 'ondisk_log_size': 7043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 915, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28823552, 'num_objects': 7037, 'num_object_clones': 0, 'num_object_copies': 14074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 27, 'num_objects_unfound': 0, 'num_objects_dirty': 7037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4946, 'num_write_kb': 19784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3453, 'num_bytes_recovered': 14135296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 27}, {'shards': '2', 'objects': 7010}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7221", 'reported_seq': 7346, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.326195+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:15:54.326195+0000', 'last_peered': '2024-09-11T05:15:54.326195+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:15:54.326195+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 7221, 'log_dups_size': 0, 'ondisk_log_size': 7221, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29569024, 'num_objects': 7219, 'num_object_clones': 0, 'num_object_copies': 14438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7219, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4254, 'num_write_kb': 17016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3596}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6990", 'reported_seq': 7119, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.257848+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:15:54.257848+0000', 'last_peered': '2024-09-11T05:15:54.257848+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:15:54.257848+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 6990, 'log_dups_size': 0, 'ondisk_log_size': 6990, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28598272, 'num_objects': 6982, 'num_object_clones': 0, 'num_object_copies': 13964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3856, 'num_write_kb': 15424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3487}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7030", 'reported_seq': 7160, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.311101+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:15:54.311101+0000', 'last_peered': '2024-09-11T05:15:54.311101+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:15:54.311101+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 7030, 'log_dups_size': 0, 'ondisk_log_size': 7030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28778496, 'num_objects': 7026, 'num_object_clones': 0, 'num_object_copies': 14052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3788, 'num_write_kb': 15152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3482}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:01.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7015", 'reported_seq': 7154, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:15:54.251096+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:15:54.251096+0000', 'last_peered': '2024-09-11T05:15:54.251096+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:15:54.251096+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 7015, 'log_dups_size': 0, 'ondisk_log_size': 7015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28684905, 'num_objects': 7004, 'num_object_clones': 0, 'num_object_copies': 14008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4022, 'num_write_kb': 16088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3489}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:01.396 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:02.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 149 20 117293 117273 3.07307 1.89453 0.0299627 0.0254141 2024-09-11T05:16:02.350 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:02.350 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/232402 objects degraded (15.246%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":240,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116201,"data_bytes":475947644,"bytes_used":1581002752,"bytes_avail":288329289728,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":232402,"degraded_ratio":0.15245565872927083,"misplaced_objects":3518,"misplaced_total":232402,"misplaced_ratio":0.015137563360039932,"recovering_objects_per_sec":46,"recovering_bytes_per_sec":191958,"recovering_keys_per_sec":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"write_bytes_sec":2496487,"read_op_per_sec":0,"write_op_per_sec":609},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:02.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:02.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:03.133 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 150 20 117775 117755 3.06513 1.88281 0.0424615 0.0254794 2024-09-11T05:16:03.293 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:03.294 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:03.343 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-11T05:16:01.412522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475947644,"num_objects":116201,"num_object_clones":0,"num_object_copies":232402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3518,"num_objects_unfound":0,"num_objects_dirty":116201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79346,"num_write_kb":317378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3562,"num_bytes_recovered":14577683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116283,"ondisk_log_size":116283,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1543948,"kb_used_data":939416,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":281571572,"statfs":{"total":289910292480,"available":288329289728,"internally_reserved":0,"allocated":961961984,"data_stored":961130083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":618974209},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,2,4,4,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19976192,"num_objects":4877,"num_object_clones":0,"num_object_copies":9754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3600,"num_objects_misplaced":3223,"num_objects_unfound":0,"num_objects_dirty":4877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4877,"num_write_kb":19508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001718"},"pg_stats":[{"pgid":"2.f","version":"45'7353","reported_seq":7431,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.335244+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:15:59.335244+0000","last_peered":"2024-09-11T05:15:59.335244+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:15:59.335244+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7353,"log_dups_size":0,"ondisk_log_size":7353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7353,"num_write_kb":29412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3678},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7364","reported_seq":7466,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:00.894524+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:00.894524+0000","last_peered":"2024-09-11T05:16:00.894524+0000","last_clean":"2024-09-11T05:16:00.894524+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:00.894524+0000","last_undegraded":"2024-09-11T05:16:00.894524+0000","last_fullsized":"2024-09-11T05:16:00.894524+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7364,"log_dups_size":0,"ondisk_log_size":7364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5762,"num_write_kb":23048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7226","reported_seq":7351,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:00.829799+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:00.829799+0000","last_peered":"2024-09-11T05:16:00.829799+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:00.829799+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5629,"num_write_kb":22516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3728}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7357","reported_seq":7428,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:00.873159+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:00.873159+0000","last_peered":"2024-09-11T05:16:00.873159+0000","last_clean":"2024-09-11T05:16:00.873159+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:00.873159+0000","last_undegraded":"2024-09-11T05:16:00.873159+0000","last_fullsized":"2024-09-11T05:16:00.873159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7357,"log_dups_size":0,"ondisk_log_size":7357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30134272,"num_objects":7357,"num_object_clones":0,"num_object_copies":14714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7357,"num_write_kb":29428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7356","reported_seq":7485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:59.354115+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:15:59.354115+0000","last_peered":"2024-09-11T05:15:59.354115+0000","last_clean":"2024-09-11T05:15:59.354115+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:15:59.354115+0000","last_undegraded":"2024-09-11T05:15:59.354115+0000","last_fullsized":"2024-09-11T05:15:59.354115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30101504,"num_objects":7349,"num_object_clones":0,"num_object_copies":14698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4190,"num_write_kb":16760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7319","reported_seq":7628,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.361580+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:15:59.361580+0000","last_peered":"2024-09-11T05:15:59.361580+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:15:59.361580+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7319,"log_dups_size":0,"ondisk_log_size":7319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29958144,"num_objects":7314,"num_object_clones":0,"num_object_copies":14628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3518,"num_objects_unfound":0,"num_objects_dirty":7314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3863,"num_write_kb":15452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3794},{"shards":"0,2","objects":3518}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7345","reported_seq":7477,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.380411+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:15:59.380411+0000","last_peered":"2024-09-11T05:15:59.380411+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:15:59.380411+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7345,"log_dups_size":0,"ondisk_log_size":7345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30060544,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4510,"num_write_kb":18040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3763},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7183","reported_seq":7306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:59.367950+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:15:59.367950+0000","last_peered":"2024-09-11T05:15:59.367950+0000","last_clean":"2024-09-11T05:15:59.367950+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:15:59.367950+0000","last_undegraded":"2024-09-11T05:15:59.367950+0000","last_fullsized":"2024-09-11T05:15:59.367950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7183,"log_dups_size":0,"ondisk_log_size":7183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29413376,"num_objects":7181,"num_object_clones":0,"num_object_copies":14362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4486,"num_write_kb":17944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.774157+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:14:13.774157+0000","last_peered":"2024-09-11T05:14:13.774157+0000","last_clean":"2024-09-11T05:14:13.774157+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:14:13.774157+0000","last_undegraded":"2024-09-11T05:14:13.774157+0000","last_fullsized":"2024-09-11T05:14:13.774157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7339","reported_seq":7470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:00.836478+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:00.836478+0000","last_peered":"2024-09-11T05:16:00.836478+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:00.836478+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7339,"log_dups_size":0,"ondisk_log_size":7339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5407,"num_write_kb":21628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7105","reported_seq":7268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.375444+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:15:59.375444+0000","last_peered":"2024-09-11T05:15:59.375444+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:15:59.375444+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7105,"log_dups_size":0,"ondisk_log_size":7105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29069312,"num_objects":7097,"num_object_clones":0,"num_object_copies":14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3858,"num_write_kb":15432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7337","reported_seq":7454,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:00.793812+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:00.793812+0000","last_peered":"2024-09-11T05:16:00.793812+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:00.793812+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5366,"num_write_kb":21464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.833112+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:14:13.833112+0000","last_peered":"2024-09-11T05:14:13.833112+0000","last_clean":"2024-09-11T05:14:13.833112+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:14:13.833112+0000","last_undegraded":"2024-09-11T05:14:13.833112+0000","last_fullsized":"2024-09-11T05:14:13.833112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7180","reported_seq":14265,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:00.856352+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:00.856352+0000","last_peered":"2024-09-11T05:16:00.856352+0000","last_clean":"2024-09-11T05:16:00.856352+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:00.856352+0000","last_undegraded":"2024-09-11T05:16:00.856352+0000","last_fullsized":"2024-09-11T05:16:00.856352+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7180,"log_dups_size":0,"ondisk_log_size":7180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5083,"num_write_kb":20332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7360","reported_seq":7491,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.394158+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:15:59.394158+0000","last_peered":"2024-09-11T05:15:59.394158+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:15:59.394158+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7360,"log_dups_size":0,"ondisk_log_size":7360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4393,"num_write_kb":17572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3735},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.980958+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:14:13.980958+0000","last_peered":"2024-09-11T05:14:13.980958+0000","last_clean":"2024-09-11T05:14:13.980958+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:14:13.980958+0000","last_undegraded":"2024-09-11T05:14:13.980958+0000","last_fullsized":"2024-09-11T05:14:13.980958+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7117","reported_seq":7250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.340625+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:15:59.340625+0000","last_peered":"2024-09-11T05:15:59.340625+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:15:59.340625+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7117,"log_dups_size":0,"ondisk_log_size":7117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29118464,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3983,"num_write_kb":15932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3614},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7182","reported_seq":7316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.394498+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:15:59.394498+0000","last_peered":"2024-09-11T05:15:59.394498+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:15:59.394498+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29401088,"num_objects":7178,"num_object_clones":0,"num_object_copies":14356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3940,"num_write_kb":15760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3634},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.983328+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:14:13.983328+0000","last_peered":"2024-09-11T05:14:13.983328+0000","last_clean":"2024-09-11T05:14:13.983328+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:14:13.983328+0000","last_undegraded":"2024-09-11T05:14:13.983328+0000","last_fullsized":"2024-09-11T05:14:13.983328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T05:14:13.750697+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:14:13.750697+0000","last_peered":"2024-09-11T05:14:13.750697+0000","last_clean":"2024-09-11T05:14:13.750697+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:14:13.750697+0000","last_undegraded":"2024-09-11T05:14:13.750697+0000","last_fullsized":"2024-09-11T05:14:13.750697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7157","reported_seq":7300,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:15:59.349368+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:15:59.349368+0000","last_peered":"2024-09-11T05:15:59.349368+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:15:59.349368+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7157,"log_dups_size":0,"ondisk_log_size":7157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29266537,"num_objects":7146,"num_object_clones":0,"num_object_copies":14292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4164,"num_write_kb":16656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3631},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475947625,"num_objects":116199,"num_object_clones":0,"num_object_copies":232398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3518,"num_objects_unfound":0,"num_objects_dirty":116199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79344,"num_write_kb":317376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3561,"num_bytes_recovered":14577664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":961818624,"data_stored":961738607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116280,"ondisk_log_size":116280,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519776,"kb_used_data":347856,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93852064,"statfs":{"total":96636764160,"available":96104513536,"internally_reserved":0,"allocated":356204544,"data_stored":355949690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349664,"kb_used_data":187472,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94022176,"statfs":{"total":96636764160,"available":96278708224,"internally_reserved":0,"allocated":191971328,"data_stored":191653212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":166063787},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215146,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674508,"kb_used_data":404088,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93697332,"statfs":{"total":96636764160,"available":95946067968,"internally_reserved":0,"allocated":413786112,"data_stored":413527181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":414023680,"data_stored":414020201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":191930368,"data_stored":191857309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355864576,"data_stored":355861097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:04.133 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 151 20 118291 118271 3.05819 2.01562 0.0100928 0.0255384 2024-09-11T05:16:05.133 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 152 20 118811 118791 3.05143 2.03125 0.0599252 0.0255951 2024-09-11T05:16:06.133 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 153 20 119328 119308 3.04469 2.01953 0.0402125 0.0256497 2024-09-11T05:16:06.345 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:07.133 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 154 20 119811 119791 3.03717 1.88672 0.0465237 0.0257147 2024-09-11T05:16:07.258 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:07.259 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:07.307 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-11T05:16:05.413432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":482419324,"num_objects":117781,"num_object_clones":0,"num_object_copies":235562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":117781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80926,"num_write_kb":323698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3626,"num_bytes_recovered":14839827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117863,"ondisk_log_size":117863,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1554532,"kb_used_data":954164,"kb_used_omap":12,"kb_used_meta":600307,"kb_avail":281560988,"statfs":{"total":289910292480,"available":288318451712,"internally_reserved":0,"allocated":977063936,"data_stored":976221640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":614714369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,8,14,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15970304,"num_objects":3899,"num_object_clones":0,"num_object_copies":7798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3600,"num_objects_misplaced":3427,"num_objects_unfound":0,"num_objects_dirty":3899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3899,"num_write_kb":15596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":700416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.867556"},"pg_stats":[{"pgid":"2.f","version":"45'7522","reported_seq":7598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.454769+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:04.454769+0000","last_peered":"2024-09-11T05:16:04.454769+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:04.454769+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7522,"log_dups_size":0,"ondisk_log_size":7522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7364","reported_seq":7466,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:00.894524+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:00.894524+0000","last_peered":"2024-09-11T05:16:00.894524+0000","last_clean":"2024-09-11T05:16:00.894524+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:00.894524+0000","last_undegraded":"2024-09-11T05:16:00.894524+0000","last_fullsized":"2024-09-11T05:16:00.894524+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7364,"log_dups_size":0,"ondisk_log_size":7364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5762,"num_write_kb":23048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7226","reported_seq":7351,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:00.829799+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:00.829799+0000","last_peered":"2024-09-11T05:16:00.829799+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:00.829799+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5629,"num_write_kb":22516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3728}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7357","reported_seq":7428,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:00.873159+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:00.873159+0000","last_peered":"2024-09-11T05:16:00.873159+0000","last_clean":"2024-09-11T05:16:00.873159+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:00.873159+0000","last_undegraded":"2024-09-11T05:16:00.873159+0000","last_fullsized":"2024-09-11T05:16:00.873159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7357,"log_dups_size":0,"ondisk_log_size":7357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30134272,"num_objects":7357,"num_object_clones":0,"num_object_copies":14714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7357,"num_write_kb":29428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7514","reported_seq":7643,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:04.279526+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:04.279526+0000","last_peered":"2024-09-11T05:16:04.279526+0000","last_clean":"2024-09-11T05:16:04.279526+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:04.279526+0000","last_undegraded":"2024-09-11T05:16:04.279526+0000","last_fullsized":"2024-09-11T05:16:04.279526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7514,"log_dups_size":0,"ondisk_log_size":7514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30748672,"num_objects":7507,"num_object_clones":0,"num_object_copies":15014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4348,"num_write_kb":17392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7484","reported_seq":7921,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.444932+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:04.444932+0000","last_peered":"2024-09-11T05:16:04.444932+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:04.444932+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7484,"log_dups_size":0,"ondisk_log_size":7484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30633984,"num_objects":7479,"num_object_clones":0,"num_object_copies":14958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":7479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4028,"num_write_kb":16112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4023},{"shards":"0,2","objects":3454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7498","reported_seq":7630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.393764+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:04.393764+0000","last_peered":"2024-09-11T05:16:04.393764+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:04.393764+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687232,"num_objects":7492,"num_object_clones":0,"num_object_copies":14984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4663,"num_write_kb":18652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3916},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7338","reported_seq":7461,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:04.405279+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:04.405279+0000","last_peered":"2024-09-11T05:16:04.405279+0000","last_clean":"2024-09-11T05:16:04.405279+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:04.405279+0000","last_undegraded":"2024-09-11T05:16:04.405279+0000","last_fullsized":"2024-09-11T05:16:04.405279+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048256,"num_objects":7336,"num_object_clones":0,"num_object_copies":14672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4641,"num_write_kb":18564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569886+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569886+0000","last_peered":"2024-09-11T05:15:57.569886+0000","last_clean":"2024-09-11T05:15:57.569886+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569886+0000","last_undegraded":"2024-09-11T05:15:57.569886+0000","last_fullsized":"2024-09-11T05:15:57.569886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7339","reported_seq":7470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:00.836478+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:00.836478+0000","last_peered":"2024-09-11T05:16:00.836478+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:00.836478+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7339,"log_dups_size":0,"ondisk_log_size":7339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5407,"num_write_kb":21628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7270","reported_seq":7434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.351476+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:04.351476+0000","last_peered":"2024-09-11T05:16:04.351476+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:04.351476+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7270,"log_dups_size":0,"ondisk_log_size":7270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29745152,"num_objects":7262,"num_object_clones":0,"num_object_copies":14524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":16092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3820},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7337","reported_seq":7454,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:00.793812+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:00.793812+0000","last_peered":"2024-09-11T05:16:00.793812+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:00.793812+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5366,"num_write_kb":21464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570072+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570072+0000","last_peered":"2024-09-11T05:15:57.570072+0000","last_clean":"2024-09-11T05:15:57.570072+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570072+0000","last_undegraded":"2024-09-11T05:15:57.570072+0000","last_fullsized":"2024-09-11T05:15:57.570072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7180","reported_seq":14265,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:00.856352+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:00.856352+0000","last_peered":"2024-09-11T05:16:00.856352+0000","last_clean":"2024-09-11T05:16:00.856352+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:00.856352+0000","last_undegraded":"2024-09-11T05:16:00.856352+0000","last_fullsized":"2024-09-11T05:16:00.856352+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7180,"log_dups_size":0,"ondisk_log_size":7180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5083,"num_write_kb":20332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7512","reported_seq":7641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.449678+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:04.449678+0000","last_peered":"2024-09-11T05:16:04.449678+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:04.449678+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30760960,"num_objects":7510,"num_object_clones":0,"num_object_copies":15020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3887},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570756+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570756+0000","last_peered":"2024-09-11T05:15:57.570756+0000","last_clean":"2024-09-11T05:15:57.570756+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570756+0000","last_undegraded":"2024-09-11T05:15:57.570756+0000","last_fullsized":"2024-09-11T05:15:57.570756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7276","reported_seq":7409,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.382001+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:04.382001+0000","last_peered":"2024-09-11T05:16:04.382001+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:04.382001+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7276,"log_dups_size":0,"ondisk_log_size":7276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29769728,"num_objects":7268,"num_object_clones":0,"num_object_copies":14536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4142,"num_write_kb":16568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3773},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7345","reported_seq":7478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.438170+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:04.438170+0000","last_peered":"2024-09-11T05:16:04.438170+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:04.438170+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7345,"log_dups_size":0,"ondisk_log_size":7345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30068736,"num_objects":7341,"num_object_clones":0,"num_object_copies":14682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570579+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570579+0000","last_peered":"2024-09-11T05:15:57.570579+0000","last_clean":"2024-09-11T05:15:57.570579+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570579+0000","last_undegraded":"2024-09-11T05:15:57.570579+0000","last_fullsized":"2024-09-11T05:15:57.570579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7298","reported_seq":7441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.385505+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:04.385505+0000","last_peered":"2024-09-11T05:16:04.385505+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:04.385505+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7298,"log_dups_size":0,"ondisk_log_size":7298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29844073,"num_objects":7287,"num_object_clones":0,"num_object_copies":14574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":17220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3772},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":482419305,"num_objects":117779,"num_object_clones":0,"num_object_copies":235558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":117779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80924,"num_write_kb":323696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3625,"num_bytes_recovered":14839808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":977031168,"data_stored":976951151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117860,"ondisk_log_size":117860,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519776,"kb_used_data":347856,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93852064,"statfs":{"total":96636764160,"available":96104513536,"internally_reserved":0,"allocated":356204544,"data_stored":355949690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348596,"kb_used_data":194724,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94023244,"statfs":{"total":96636764160,"available":96279801856,"internally_reserved":0,"allocated":199397376,"data_stored":199068865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157544107},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215147,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686160,"kb_used_data":411584,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93685680,"statfs":{"total":96636764160,"available":95934136320,"internally_reserved":0,"allocated":421462016,"data_stored":421203085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421715968,"data_stored":421712489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199450624,"data_stored":199377565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355864576,"data_stored":355861097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:07.308 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:08.133 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 155 20 120292 120272 3.0297 1.87891 0.0228969 0.025777 2024-09-11T05:16:08.173 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:08.176 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:08.207 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-11T05:16:07.413918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486290044,"num_objects":118726,"num_object_clones":0,"num_object_copies":237452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":118726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81871,"num_write_kb":327478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3626,"num_bytes_recovered":14839827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118808,"ondisk_log_size":118808,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1574556,"kb_used_data":965868,"kb_used_omap":12,"kb_used_meta":608627,"kb_avail":281540964,"statfs":{"total":289910292480,"available":288297947136,"internally_reserved":0,"allocated":989048832,"data_stored":988206536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":623234049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,7,15,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19841024,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3600,"num_objects_misplaced":3427,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4844,"num_write_kb":19376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":700416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002165"},"pg_stats":[{"pgid":"2.f","version":"45'7522","reported_seq":7598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.454769+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:04.454769+0000","last_peered":"2024-09-11T05:16:04.454769+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:04.454769+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7522,"log_dups_size":0,"ondisk_log_size":7522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7506","reported_seq":7608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:05.750453+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:05.750453+0000","last_peered":"2024-09-11T05:16:05.750453+0000","last_clean":"2024-09-11T05:16:05.750453+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:05.750453+0000","last_undegraded":"2024-09-11T05:16:05.750453+0000","last_fullsized":"2024-09-11T05:16:05.750453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7506,"log_dups_size":0,"ondisk_log_size":7506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720000,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5904,"num_write_kb":23616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7391","reported_seq":7516,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:05.860899+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:05.860899+0000","last_peered":"2024-09-11T05:16:05.860899+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:05.860899+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7391,"log_dups_size":0,"ondisk_log_size":7391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257152,"num_objects":7387,"num_object_clones":0,"num_object_copies":14774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5794,"num_write_kb":23176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7515","reported_seq":7586,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:05.857386+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:05.857386+0000","last_peered":"2024-09-11T05:16:05.857386+0000","last_clean":"2024-09-11T05:16:05.857386+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:05.857386+0000","last_undegraded":"2024-09-11T05:16:05.857386+0000","last_fullsized":"2024-09-11T05:16:05.857386+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7515,"log_dups_size":0,"ondisk_log_size":7515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":30060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7514","reported_seq":7643,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:04.279526+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:04.279526+0000","last_peered":"2024-09-11T05:16:04.279526+0000","last_clean":"2024-09-11T05:16:04.279526+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:04.279526+0000","last_undegraded":"2024-09-11T05:16:04.279526+0000","last_fullsized":"2024-09-11T05:16:04.279526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7514,"log_dups_size":0,"ondisk_log_size":7514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30748672,"num_objects":7507,"num_object_clones":0,"num_object_copies":15014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4348,"num_write_kb":17392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7484","reported_seq":7921,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.444932+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:04.444932+0000","last_peered":"2024-09-11T05:16:04.444932+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:04.444932+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7484,"log_dups_size":0,"ondisk_log_size":7484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30633984,"num_objects":7479,"num_object_clones":0,"num_object_copies":14958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":7479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4028,"num_write_kb":16112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4023},{"shards":"0,2","objects":3454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7498","reported_seq":7630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.393764+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:04.393764+0000","last_peered":"2024-09-11T05:16:04.393764+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:04.393764+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687232,"num_objects":7492,"num_object_clones":0,"num_object_copies":14984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4663,"num_write_kb":18652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3916},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7338","reported_seq":7461,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:04.405279+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:04.405279+0000","last_peered":"2024-09-11T05:16:04.405279+0000","last_clean":"2024-09-11T05:16:04.405279+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:04.405279+0000","last_undegraded":"2024-09-11T05:16:04.405279+0000","last_fullsized":"2024-09-11T05:16:04.405279+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048256,"num_objects":7336,"num_object_clones":0,"num_object_copies":14672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4641,"num_write_kb":18564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569886+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569886+0000","last_peered":"2024-09-11T05:15:57.569886+0000","last_clean":"2024-09-11T05:15:57.569886+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569886+0000","last_undegraded":"2024-09-11T05:15:57.569886+0000","last_fullsized":"2024-09-11T05:15:57.569886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7491","reported_seq":7622,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:05.872570+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:05.872570+0000","last_peered":"2024-09-11T05:16:05.872570+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:05.872570+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7491,"log_dups_size":0,"ondisk_log_size":7491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30654464,"num_objects":7484,"num_object_clones":0,"num_object_copies":14968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5559,"num_write_kb":22236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3959}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7270","reported_seq":7434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.351476+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:04.351476+0000","last_peered":"2024-09-11T05:16:04.351476+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:04.351476+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7270,"log_dups_size":0,"ondisk_log_size":7270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29745152,"num_objects":7262,"num_object_clones":0,"num_object_copies":14524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":16092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3820},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7501","reported_seq":7615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:05.860055+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:05.860055+0000","last_peered":"2024-09-11T05:16:05.860055+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:05.860055+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30703616,"num_objects":7496,"num_object_clones":0,"num_object_copies":14992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5530,"num_write_kb":22120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3974}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570072+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570072+0000","last_peered":"2024-09-11T05:15:57.570072+0000","last_clean":"2024-09-11T05:15:57.570072+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570072+0000","last_undegraded":"2024-09-11T05:15:57.570072+0000","last_fullsized":"2024-09-11T05:15:57.570072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7344","reported_seq":14429,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:05.851302+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:05.851302+0000","last_peered":"2024-09-11T05:16:05.851302+0000","last_clean":"2024-09-11T05:16:05.851302+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:05.851302+0000","last_undegraded":"2024-09-11T05:16:05.851302+0000","last_fullsized":"2024-09-11T05:16:05.851302+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7512","reported_seq":7641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.449678+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:04.449678+0000","last_peered":"2024-09-11T05:16:04.449678+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:04.449678+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30760960,"num_objects":7510,"num_object_clones":0,"num_object_copies":15020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3887},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570756+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570756+0000","last_peered":"2024-09-11T05:15:57.570756+0000","last_clean":"2024-09-11T05:15:57.570756+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570756+0000","last_undegraded":"2024-09-11T05:15:57.570756+0000","last_fullsized":"2024-09-11T05:15:57.570756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7276","reported_seq":7409,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.382001+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:04.382001+0000","last_peered":"2024-09-11T05:16:04.382001+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:04.382001+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7276,"log_dups_size":0,"ondisk_log_size":7276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29769728,"num_objects":7268,"num_object_clones":0,"num_object_copies":14536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4142,"num_write_kb":16568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3773},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7345","reported_seq":7478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.438170+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:04.438170+0000","last_peered":"2024-09-11T05:16:04.438170+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:04.438170+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7345,"log_dups_size":0,"ondisk_log_size":7345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30068736,"num_objects":7341,"num_object_clones":0,"num_object_copies":14682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570579+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570579+0000","last_peered":"2024-09-11T05:15:57.570579+0000","last_clean":"2024-09-11T05:15:57.570579+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570579+0000","last_undegraded":"2024-09-11T05:15:57.570579+0000","last_fullsized":"2024-09-11T05:15:57.570579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7298","reported_seq":7441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.385505+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:04.385505+0000","last_peered":"2024-09-11T05:16:04.385505+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:04.385505+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7298,"log_dups_size":0,"ondisk_log_size":7298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29844073,"num_objects":7287,"num_object_clones":0,"num_object_copies":14574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":17220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3772},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486290025,"num_objects":118724,"num_object_clones":0,"num_object_copies":237448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":118724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81869,"num_write_kb":327476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3625,"num_bytes_recovered":14839808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":990367744,"data_stored":990287727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118805,"ondisk_log_size":118805,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527956,"kb_used_data":351876,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93843884,"statfs":{"total":96636764160,"available":96096137216,"internally_reserved":0,"allocated":360321024,"data_stored":360066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180285099},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":360440,"kb_used_data":202408,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94011400,"statfs":{"total":96636764160,"available":96267673600,"internally_reserved":0,"allocated":207265792,"data_stored":206937281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215147,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686160,"kb_used_data":411584,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93685680,"statfs":{"total":96636764160,"available":95934136320,"internally_reserved":0,"allocated":421462016,"data_stored":421203085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421715968,"data_stored":421712489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207622144,"data_stored":207549085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":361029632,"data_stored":361026153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:08.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'7522", 'reported_seq': 7598, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.454769+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:04.454769+0000', 'last_peered': '2024-09-11T05:16:04.454769+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:04.454769+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 7522, 'log_dups_size': 0, 'ondisk_log_size': 7522, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30810112, 'num_objects': 7522, 'num_object_clones': 0, 'num_object_copies': 15044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7522, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7522, 'num_write_kb': 30088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3847}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'7391", 'reported_seq': 7516, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:05.860899+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:05.860899+0000', 'last_peered': '2024-09-11T05:16:05.860899+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:05.860899+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 7391, 'log_dups_size': 0, 'ondisk_log_size': 7391, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30257152, 'num_objects': 7387, 'num_object_clones': 0, 'num_object_copies': 14774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7387, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5794, 'num_write_kb': 23176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 3893}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'7484", 'reported_seq': 7921, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.444932+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:04.444932+0000', 'last_peered': '2024-09-11T05:16:04.444932+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:04.444932+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 7484, 'log_dups_size': 0, 'ondisk_log_size': 7484, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30633984, 'num_objects': 7479, 'num_object_clones': 0, 'num_object_copies': 14958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3454, 'num_objects_unfound': 0, 'num_objects_dirty': 7479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4028, 'num_write_kb': 16112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 589824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4023}, {'shards': '0,2', 'objects': 3454}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'7498", 'reported_seq': 7630, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.393764+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:04.393764+0000', 'last_peered': '2024-09-11T05:16:04.393764+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:04.393764+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 7498, 'log_dups_size': 0, 'ondisk_log_size': 7498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30687232, 'num_objects': 7492, 'num_object_clones': 0, 'num_object_copies': 14984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4663, 'num_write_kb': 18652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3916}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:08.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'7491", 'reported_seq': 7622, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:05.872570+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:05.872570+0000', 'last_peered': '2024-09-11T05:16:05.872570+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:05.872570+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 7491, 'log_dups_size': 0, 'ondisk_log_size': 7491, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30654464, 'num_objects': 7484, 'num_object_clones': 0, 'num_object_copies': 14968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7484, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5559, 'num_write_kb': 22236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 3959}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'7270", 'reported_seq': 7434, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.351476+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:04.351476+0000', 'last_peered': '2024-09-11T05:16:04.351476+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:04.351476+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 7270, 'log_dups_size': 0, 'ondisk_log_size': 7270, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29745152, 'num_objects': 7262, 'num_object_clones': 0, 'num_object_copies': 14524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4023, 'num_write_kb': 16092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3820}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'7501", 'reported_seq': 7615, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:05.860055+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:05.860055+0000', 'last_peered': '2024-09-11T05:16:05.860055+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:05.860055+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 7501, 'log_dups_size': 0, 'ondisk_log_size': 7501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30703616, 'num_objects': 7496, 'num_object_clones': 0, 'num_object_copies': 14992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5530, 'num_write_kb': 22120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 3974}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'7512", 'reported_seq': 7641, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.449678+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:04.449678+0000', 'last_peered': '2024-09-11T05:16:04.449678+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:04.449678+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 7512, 'log_dups_size': 0, 'ondisk_log_size': 7512, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30760960, 'num_objects': 7510, 'num_object_clones': 0, 'num_object_copies': 15020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4545, 'num_write_kb': 18180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3887}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7276", 'reported_seq': 7409, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.382001+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:04.382001+0000', 'last_peered': '2024-09-11T05:16:04.382001+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:04.382001+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 7276, 'log_dups_size': 0, 'ondisk_log_size': 7276, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29769728, 'num_objects': 7268, 'num_object_clones': 0, 'num_object_copies': 14536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4142, 'num_write_kb': 16568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3773}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:08.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'7345", 'reported_seq': 7478, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.438170+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:04.438170+0000', 'last_peered': '2024-09-11T05:16:04.438170+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:04.438170+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 7345, 'log_dups_size': 0, 'ondisk_log_size': 7345, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30068736, 'num_objects': 7341, 'num_object_clones': 0, 'num_object_copies': 14682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4103, 'num_write_kb': 16412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3797}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:08.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'7298", 'reported_seq': 7441, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:04.385505+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:04.385505+0000', 'last_peered': '2024-09-11T05:16:04.385505+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:04.385505+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 7298, 'log_dups_size': 0, 'ondisk_log_size': 7298, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29844073, 'num_objects': 7287, 'num_object_clones': 0, 'num_object_copies': 14574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4305, 'num_write_kb': 17220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3772}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:08.211 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:09.134 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 156 20 120774 120754 3.02233 1.88281 0.0321037 0.0258396 2024-09-11T05:16:09.156 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:09.156 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/237452 objects degraded (14.921%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":246,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118726,"data_bytes":486290044,"bytes_used":1612345344,"bytes_avail":288297947136,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":237452,"degraded_ratio":0.14921331469096913,"misplaced_objects":3454,"misplaced_total":237452,"misplaced_ratio":0.014546097737648029,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70026,"recovering_keys_per_sec":0,"num_objects_recovered":171,"num_bytes_recovered":700416,"num_keys_recovered":0,"write_bytes_sec":1983672,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:09.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:09.241 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:09.957 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:10.032 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-11T05:16:09.414450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486290044,"num_objects":118726,"num_object_clones":0,"num_object_copies":237452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":118726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81871,"num_write_kb":327478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3626,"num_bytes_recovered":14839827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118808,"ondisk_log_size":118808,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1574556,"kb_used_data":965868,"kb_used_omap":12,"kb_used_meta":608627,"kb_avail":281540964,"statfs":{"total":289910292480,"available":288297947136,"internally_reserved":0,"allocated":989048832,"data_stored":988206536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":623234049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,7,15,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19841024,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3600,"num_objects_misplaced":3427,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4844,"num_write_kb":19376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":700416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.868273"},"pg_stats":[{"pgid":"2.f","version":"45'7522","reported_seq":7598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.454769+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:04.454769+0000","last_peered":"2024-09-11T05:16:04.454769+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:04.454769+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7522,"log_dups_size":0,"ondisk_log_size":7522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7506","reported_seq":7608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:05.750453+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:05.750453+0000","last_peered":"2024-09-11T05:16:05.750453+0000","last_clean":"2024-09-11T05:16:05.750453+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:05.750453+0000","last_undegraded":"2024-09-11T05:16:05.750453+0000","last_fullsized":"2024-09-11T05:16:05.750453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7506,"log_dups_size":0,"ondisk_log_size":7506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720000,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5904,"num_write_kb":23616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7391","reported_seq":7516,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:05.860899+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:05.860899+0000","last_peered":"2024-09-11T05:16:05.860899+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:05.860899+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7391,"log_dups_size":0,"ondisk_log_size":7391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257152,"num_objects":7387,"num_object_clones":0,"num_object_copies":14774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5794,"num_write_kb":23176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":3893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7515","reported_seq":7586,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:05.857386+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:05.857386+0000","last_peered":"2024-09-11T05:16:05.857386+0000","last_clean":"2024-09-11T05:16:05.857386+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:05.857386+0000","last_undegraded":"2024-09-11T05:16:05.857386+0000","last_fullsized":"2024-09-11T05:16:05.857386+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7515,"log_dups_size":0,"ondisk_log_size":7515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7515,"num_write_kb":30060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7514","reported_seq":7643,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:04.279526+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:04.279526+0000","last_peered":"2024-09-11T05:16:04.279526+0000","last_clean":"2024-09-11T05:16:04.279526+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:04.279526+0000","last_undegraded":"2024-09-11T05:16:04.279526+0000","last_fullsized":"2024-09-11T05:16:04.279526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7514,"log_dups_size":0,"ondisk_log_size":7514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30748672,"num_objects":7507,"num_object_clones":0,"num_object_copies":15014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4348,"num_write_kb":17392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7484","reported_seq":7921,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.444932+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:04.444932+0000","last_peered":"2024-09-11T05:16:04.444932+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:04.444932+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7484,"log_dups_size":0,"ondisk_log_size":7484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30633984,"num_objects":7479,"num_object_clones":0,"num_object_copies":14958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":7479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4028,"num_write_kb":16112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4023},{"shards":"0,2","objects":3454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7498","reported_seq":7630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.393764+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:04.393764+0000","last_peered":"2024-09-11T05:16:04.393764+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:04.393764+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687232,"num_objects":7492,"num_object_clones":0,"num_object_copies":14984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4663,"num_write_kb":18652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3916},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7338","reported_seq":7461,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:04.405279+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:04.405279+0000","last_peered":"2024-09-11T05:16:04.405279+0000","last_clean":"2024-09-11T05:16:04.405279+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:04.405279+0000","last_undegraded":"2024-09-11T05:16:04.405279+0000","last_fullsized":"2024-09-11T05:16:04.405279+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048256,"num_objects":7336,"num_object_clones":0,"num_object_copies":14672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4641,"num_write_kb":18564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569886+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569886+0000","last_peered":"2024-09-11T05:15:57.569886+0000","last_clean":"2024-09-11T05:15:57.569886+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569886+0000","last_undegraded":"2024-09-11T05:15:57.569886+0000","last_fullsized":"2024-09-11T05:15:57.569886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7491","reported_seq":7622,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:05.872570+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:05.872570+0000","last_peered":"2024-09-11T05:16:05.872570+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:05.872570+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7491,"log_dups_size":0,"ondisk_log_size":7491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30654464,"num_objects":7484,"num_object_clones":0,"num_object_copies":14968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5559,"num_write_kb":22236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":3959}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7270","reported_seq":7434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.351476+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:04.351476+0000","last_peered":"2024-09-11T05:16:04.351476+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:04.351476+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7270,"log_dups_size":0,"ondisk_log_size":7270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29745152,"num_objects":7262,"num_object_clones":0,"num_object_copies":14524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":16092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3820},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7501","reported_seq":7615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:05.860055+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:05.860055+0000","last_peered":"2024-09-11T05:16:05.860055+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:05.860055+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30703616,"num_objects":7496,"num_object_clones":0,"num_object_copies":14992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5530,"num_write_kb":22120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":3974}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570072+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570072+0000","last_peered":"2024-09-11T05:15:57.570072+0000","last_clean":"2024-09-11T05:15:57.570072+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570072+0000","last_undegraded":"2024-09-11T05:15:57.570072+0000","last_fullsized":"2024-09-11T05:15:57.570072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7344","reported_seq":14429,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:05.851302+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:05.851302+0000","last_peered":"2024-09-11T05:16:05.851302+0000","last_clean":"2024-09-11T05:16:05.851302+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:05.851302+0000","last_undegraded":"2024-09-11T05:16:05.851302+0000","last_fullsized":"2024-09-11T05:16:05.851302+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7512","reported_seq":7641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.449678+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:04.449678+0000","last_peered":"2024-09-11T05:16:04.449678+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:04.449678+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30760960,"num_objects":7510,"num_object_clones":0,"num_object_copies":15020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3887},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570756+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570756+0000","last_peered":"2024-09-11T05:15:57.570756+0000","last_clean":"2024-09-11T05:15:57.570756+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570756+0000","last_undegraded":"2024-09-11T05:15:57.570756+0000","last_fullsized":"2024-09-11T05:15:57.570756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7276","reported_seq":7409,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.382001+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:04.382001+0000","last_peered":"2024-09-11T05:16:04.382001+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:04.382001+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7276,"log_dups_size":0,"ondisk_log_size":7276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29769728,"num_objects":7268,"num_object_clones":0,"num_object_copies":14536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4142,"num_write_kb":16568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3773},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7345","reported_seq":7478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.438170+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:04.438170+0000","last_peered":"2024-09-11T05:16:04.438170+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:04.438170+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7345,"log_dups_size":0,"ondisk_log_size":7345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30068736,"num_objects":7341,"num_object_clones":0,"num_object_copies":14682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570579+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570579+0000","last_peered":"2024-09-11T05:15:57.570579+0000","last_clean":"2024-09-11T05:15:57.570579+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570579+0000","last_undegraded":"2024-09-11T05:15:57.570579+0000","last_fullsized":"2024-09-11T05:15:57.570579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7298","reported_seq":7441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:04.385505+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:04.385505+0000","last_peered":"2024-09-11T05:16:04.385505+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:04.385505+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7298,"log_dups_size":0,"ondisk_log_size":7298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29844073,"num_objects":7287,"num_object_clones":0,"num_object_copies":14574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":17220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3772},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486290025,"num_objects":118724,"num_object_clones":0,"num_object_copies":237448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3454,"num_objects_unfound":0,"num_objects_dirty":118724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81869,"num_write_kb":327476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3625,"num_bytes_recovered":14839808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":990367744,"data_stored":990287727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118805,"ondisk_log_size":118805,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527956,"kb_used_data":351876,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93843884,"statfs":{"total":96636764160,"available":96096137216,"internally_reserved":0,"allocated":360321024,"data_stored":360066170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180285099},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":360440,"kb_used_data":202408,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94011400,"statfs":{"total":96636764160,"available":96267673600,"internally_reserved":0,"allocated":207265792,"data_stored":206937281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161803947},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215147,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686160,"kb_used_data":411584,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93685680,"statfs":{"total":96636764160,"available":95934136320,"internally_reserved":0,"allocated":421462016,"data_stored":421203085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421715968,"data_stored":421712489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207622144,"data_stored":207549085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":361029632,"data_stored":361026153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:10.137 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 157 20 121244 121224 3.01478 1.83594 0.0784939 0.025904 2024-09-11T05:16:11.134 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 158 20 121752 121732 3.00825 1.98438 0.0512862 0.0259613 2024-09-11T05:16:12.134 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 159 20 122267 122247 3.00199 2.01172 0.0379056 0.0260165 2024-09-11T05:16:13.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:13.134 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:16:13.135854+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0260697 2024-09-11T05:16:13.134 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:16:13.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 160 20 122783 122763 2.99582 2.01562 0.0358757 0.0260697 2024-09-11T05:16:13.803 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:13.804 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:13.867 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-11T05:16:13.415460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":496325244,"num_objects":121176,"num_object_clones":0,"num_object_copies":242352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":121176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84321,"num_write_kb":337278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":15118355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121258,"ondisk_log_size":121258,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599076,"kb_used_data":986228,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281516444,"statfs":{"total":289910292480,"available":288272838656,"internally_reserved":0,"allocated":1009897472,"data_stored":1009055176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":627493889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,5,21,12,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20377600,"num_objects":4975,"num_object_clones":0,"num_object_copies":9950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-132,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4975,"num_write_kb":19900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":540672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"2.f","version":"45'7680","reported_seq":7758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.448951+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:09.448951+0000","last_peered":"2024-09-11T05:16:09.448951+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:09.448951+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7680,"log_dups_size":0,"ondisk_log_size":7680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31457280,"num_objects":7680,"num_object_clones":0,"num_object_copies":15360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7680,"num_write_kb":30720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4005},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7642","reported_seq":7744,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.854936+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:10.854936+0000","last_peered":"2024-09-11T05:16:10.854936+0000","last_clean":"2024-09-11T05:16:10.854936+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:10.854936+0000","last_undegraded":"2024-09-11T05:16:10.854936+0000","last_fullsized":"2024-09-11T05:16:10.854936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6040,"num_write_kb":24160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7554","reported_seq":7678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.878867+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:10.878867+0000","last_peered":"2024-09-11T05:16:10.878867+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:10.878867+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7554,"log_dups_size":0,"ondisk_log_size":7554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30924800,"num_objects":7550,"num_object_clones":0,"num_object_copies":15100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":23828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4056}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7659","reported_seq":7730,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.849194+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:10.849194+0000","last_peered":"2024-09-11T05:16:10.849194+0000","last_clean":"2024-09-11T05:16:10.849194+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:10.849194+0000","last_undegraded":"2024-09-11T05:16:10.849194+0000","last_fullsized":"2024-09-11T05:16:10.849194+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7659,"log_dups_size":0,"ondisk_log_size":7659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31371264,"num_objects":7659,"num_object_clones":0,"num_object_copies":15318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7659,"num_write_kb":30636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7680","reported_seq":7807,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:09.440168+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:09.440168+0000","last_peered":"2024-09-11T05:16:09.440168+0000","last_clean":"2024-09-11T05:16:09.440168+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:09.440168+0000","last_undegraded":"2024-09-11T05:16:09.440168+0000","last_fullsized":"2024-09-11T05:16:09.440168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7680,"log_dups_size":0,"ondisk_log_size":7680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7640","reported_seq":8211,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.476795+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:09.476795+0000","last_peered":"2024-09-11T05:16:09.476795+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:09.476795+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7640,"log_dups_size":0,"ondisk_log_size":7640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31272960,"num_objects":7635,"num_object_clones":0,"num_object_copies":15270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":7635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4184,"num_write_kb":16736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":868352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":3386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7642","reported_seq":7774,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.348529+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:09.348529+0000","last_peered":"2024-09-11T05:16:09.348529+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:09.348529+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4060},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7500","reported_seq":7624,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:09.454292+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:09.454292+0000","last_peered":"2024-09-11T05:16:09.454292+0000","last_clean":"2024-09-11T05:16:09.454292+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:09.454292+0000","last_undegraded":"2024-09-11T05:16:09.454292+0000","last_fullsized":"2024-09-11T05:16:09.454292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7500,"log_dups_size":0,"ondisk_log_size":7500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30711808,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":19212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7642","reported_seq":7773,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.891942+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:10.891942+0000","last_peered":"2024-09-11T05:16:10.891942+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:10.891942+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31272960,"num_objects":7635,"num_object_clones":0,"num_object_copies":15270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5710,"num_write_kb":22840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7414","reported_seq":7579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.490718+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:09.490718+0000","last_peered":"2024-09-11T05:16:09.490718+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:09.490718+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7646","reported_seq":7763,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.789592+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:10.789592+0000","last_peered":"2024-09-11T05:16:10.789592+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:10.789592+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7646,"log_dups_size":0,"ondisk_log_size":7646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31297536,"num_objects":7641,"num_object_clones":0,"num_object_copies":15282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570072+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570072+0000","last_peered":"2024-09-11T05:15:57.570072+0000","last_clean":"2024-09-11T05:15:57.570072+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570072+0000","last_undegraded":"2024-09-11T05:15:57.570072+0000","last_fullsized":"2024-09-11T05:15:57.570072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7494","reported_seq":14579,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.906192+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:10.906192+0000","last_peered":"2024-09-11T05:16:10.906192+0000","last_clean":"2024-09-11T05:16:10.906192+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:10.906192+0000","last_undegraded":"2024-09-11T05:16:10.906192+0000","last_fullsized":"2024-09-11T05:16:10.906192+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7494,"log_dups_size":0,"ondisk_log_size":7494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30670848,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":21588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7686","reported_seq":7817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.423558+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:09.423558+0000","last_peered":"2024-09-11T05:16:09.423558+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:09.423558+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7686,"log_dups_size":0,"ondisk_log_size":7686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31473664,"num_objects":7684,"num_object_clones":0,"num_object_copies":15368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4719,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570756+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570756+0000","last_peered":"2024-09-11T05:15:57.570756+0000","last_clean":"2024-09-11T05:15:57.570756+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570756+0000","last_undegraded":"2024-09-11T05:15:57.570756+0000","last_fullsized":"2024-09-11T05:15:57.570756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7430","reported_seq":7563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.417764+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:09.417764+0000","last_peered":"2024-09-11T05:16:09.417764+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:09.417764+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7430,"log_dups_size":0,"ondisk_log_size":7430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4296,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7499","reported_seq":7633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.436416+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:09.436416+0000","last_peered":"2024-09-11T05:16:09.436416+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:09.436416+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30699520,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4257,"num_write_kb":17028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3951},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570579+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570579+0000","last_peered":"2024-09-11T05:15:57.570579+0000","last_clean":"2024-09-11T05:15:57.570579+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570579+0000","last_undegraded":"2024-09-11T05:15:57.570579+0000","last_fullsized":"2024-09-11T05:15:57.570579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7447","reported_seq":7590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.362678+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:09.362678+0000","last_peered":"2024-09-11T05:16:09.362678+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:09.362678+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7447,"log_dups_size":0,"ondisk_log_size":7447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30454377,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4454,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":496325225,"num_objects":121174,"num_object_clones":0,"num_object_copies":242348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":121174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84319,"num_write_kb":337276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3693,"num_bytes_recovered":15118336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1010806784,"data_stored":1010726767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121255,"ondisk_log_size":121255,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537016,"kb_used_data":356776,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93834824,"statfs":{"total":96636764160,"available":96086859776,"internally_reserved":0,"allocated":365338624,"data_stored":365083770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184544939},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376504,"kb_used_data":210152,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93995336,"statfs":{"total":96636764160,"available":96251224064,"internally_reserved":0,"allocated":215195648,"data_stored":214867137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215148,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685556,"kb_used_data":419300,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93686284,"statfs":{"total":96636764160,"available":95934754816,"internally_reserved":0,"allocated":429363200,"data_stored":429104269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,10,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429309952,"data_stored":429306473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215560192,"data_stored":215487133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365936640,"data_stored":365933161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:13.869 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:14.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 161 20 123248 123228 2.9885 1.81641 0.0277043 0.0261346 2024-09-11T05:16:14.669 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:14.670 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:14.746 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-11T05:16:13.415460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":496325244,"num_objects":121176,"num_object_clones":0,"num_object_copies":242352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":121176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84321,"num_write_kb":337278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":15118355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121258,"ondisk_log_size":121258,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599076,"kb_used_data":986228,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281516444,"statfs":{"total":289910292480,"available":288272838656,"internally_reserved":0,"allocated":1009897472,"data_stored":1009055176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":627493889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,5,21,12,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20377600,"num_objects":4975,"num_object_clones":0,"num_object_copies":9950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-132,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4975,"num_write_kb":19900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":540672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"2.f","version":"45'7680","reported_seq":7758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.448951+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:09.448951+0000","last_peered":"2024-09-11T05:16:09.448951+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:09.448951+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7680,"log_dups_size":0,"ondisk_log_size":7680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31457280,"num_objects":7680,"num_object_clones":0,"num_object_copies":15360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7680,"num_write_kb":30720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4005},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7642","reported_seq":7744,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.854936+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:10.854936+0000","last_peered":"2024-09-11T05:16:10.854936+0000","last_clean":"2024-09-11T05:16:10.854936+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:10.854936+0000","last_undegraded":"2024-09-11T05:16:10.854936+0000","last_fullsized":"2024-09-11T05:16:10.854936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6040,"num_write_kb":24160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7554","reported_seq":7678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.878867+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:10.878867+0000","last_peered":"2024-09-11T05:16:10.878867+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:10.878867+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7554,"log_dups_size":0,"ondisk_log_size":7554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30924800,"num_objects":7550,"num_object_clones":0,"num_object_copies":15100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":23828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4056}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7659","reported_seq":7730,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.849194+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:10.849194+0000","last_peered":"2024-09-11T05:16:10.849194+0000","last_clean":"2024-09-11T05:16:10.849194+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:10.849194+0000","last_undegraded":"2024-09-11T05:16:10.849194+0000","last_fullsized":"2024-09-11T05:16:10.849194+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7659,"log_dups_size":0,"ondisk_log_size":7659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31371264,"num_objects":7659,"num_object_clones":0,"num_object_copies":15318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7659,"num_write_kb":30636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7680","reported_seq":7807,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:09.440168+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:09.440168+0000","last_peered":"2024-09-11T05:16:09.440168+0000","last_clean":"2024-09-11T05:16:09.440168+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:09.440168+0000","last_undegraded":"2024-09-11T05:16:09.440168+0000","last_fullsized":"2024-09-11T05:16:09.440168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7680,"log_dups_size":0,"ondisk_log_size":7680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7640","reported_seq":8211,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.476795+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:09.476795+0000","last_peered":"2024-09-11T05:16:09.476795+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:09.476795+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7640,"log_dups_size":0,"ondisk_log_size":7640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31272960,"num_objects":7635,"num_object_clones":0,"num_object_copies":15270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":7635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4184,"num_write_kb":16736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":868352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":3386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7642","reported_seq":7774,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.348529+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:09.348529+0000","last_peered":"2024-09-11T05:16:09.348529+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:09.348529+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4060},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7500","reported_seq":7624,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:09.454292+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:09.454292+0000","last_peered":"2024-09-11T05:16:09.454292+0000","last_clean":"2024-09-11T05:16:09.454292+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:09.454292+0000","last_undegraded":"2024-09-11T05:16:09.454292+0000","last_fullsized":"2024-09-11T05:16:09.454292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7500,"log_dups_size":0,"ondisk_log_size":7500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30711808,"num_objects":7498,"num_object_clones":0,"num_object_copies":14996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4803,"num_write_kb":19212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7642","reported_seq":7773,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.891942+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:10.891942+0000","last_peered":"2024-09-11T05:16:10.891942+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:10.891942+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31272960,"num_objects":7635,"num_object_clones":0,"num_object_copies":15270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5710,"num_write_kb":22840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7414","reported_seq":7579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.490718+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:09.490718+0000","last_peered":"2024-09-11T05:16:09.490718+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:09.490718+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7646","reported_seq":7763,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.789592+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:10.789592+0000","last_peered":"2024-09-11T05:16:10.789592+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:10.789592+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7646,"log_dups_size":0,"ondisk_log_size":7646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31297536,"num_objects":7641,"num_object_clones":0,"num_object_copies":15282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570072+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570072+0000","last_peered":"2024-09-11T05:15:57.570072+0000","last_clean":"2024-09-11T05:15:57.570072+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570072+0000","last_undegraded":"2024-09-11T05:15:57.570072+0000","last_fullsized":"2024-09-11T05:15:57.570072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7494","reported_seq":14579,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.906192+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:10.906192+0000","last_peered":"2024-09-11T05:16:10.906192+0000","last_clean":"2024-09-11T05:16:10.906192+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:10.906192+0000","last_undegraded":"2024-09-11T05:16:10.906192+0000","last_fullsized":"2024-09-11T05:16:10.906192+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7494,"log_dups_size":0,"ondisk_log_size":7494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30670848,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":21588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7686","reported_seq":7817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.423558+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:09.423558+0000","last_peered":"2024-09-11T05:16:09.423558+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:09.423558+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7686,"log_dups_size":0,"ondisk_log_size":7686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31473664,"num_objects":7684,"num_object_clones":0,"num_object_copies":15368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4719,"num_write_kb":18876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570756+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570756+0000","last_peered":"2024-09-11T05:15:57.570756+0000","last_clean":"2024-09-11T05:15:57.570756+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570756+0000","last_undegraded":"2024-09-11T05:15:57.570756+0000","last_fullsized":"2024-09-11T05:15:57.570756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7430","reported_seq":7563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.417764+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:09.417764+0000","last_peered":"2024-09-11T05:16:09.417764+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:09.417764+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7430,"log_dups_size":0,"ondisk_log_size":7430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4296,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7499","reported_seq":7633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.436416+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:09.436416+0000","last_peered":"2024-09-11T05:16:09.436416+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:09.436416+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30699520,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4257,"num_write_kb":17028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3951},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570579+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570579+0000","last_peered":"2024-09-11T05:15:57.570579+0000","last_clean":"2024-09-11T05:15:57.570579+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570579+0000","last_undegraded":"2024-09-11T05:15:57.570579+0000","last_fullsized":"2024-09-11T05:15:57.570579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7447","reported_seq":7590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:09.362678+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:09.362678+0000","last_peered":"2024-09-11T05:16:09.362678+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:09.362678+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7447,"log_dups_size":0,"ondisk_log_size":7447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30454377,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4454,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":496325225,"num_objects":121174,"num_object_clones":0,"num_object_copies":242348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3386,"num_objects_unfound":0,"num_objects_dirty":121174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84319,"num_write_kb":337276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3693,"num_bytes_recovered":15118336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1010806784,"data_stored":1010726767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121255,"ondisk_log_size":121255,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537016,"kb_used_data":356776,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93834824,"statfs":{"total":96636764160,"available":96086859776,"internally_reserved":0,"allocated":365338624,"data_stored":365083770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184544939},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376504,"kb_used_data":210152,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93995336,"statfs":{"total":96636764160,"available":96251224064,"internally_reserved":0,"allocated":215195648,"data_stored":214867137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215148,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685556,"kb_used_data":419300,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93686284,"statfs":{"total":96636764160,"available":95934754816,"internally_reserved":0,"allocated":429363200,"data_stored":429104269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,10,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429309952,"data_stored":429306473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215560192,"data_stored":215487133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365936640,"data_stored":365933161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'7680", 'reported_seq': 7758, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.448951+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:09.448951+0000', 'last_peered': '2024-09-11T05:16:09.448951+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:09.448951+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 7680, 'log_dups_size': 0, 'ondisk_log_size': 7680, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31457280, 'num_objects': 7680, 'num_object_clones': 0, 'num_object_copies': 15360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7680, 'num_write_kb': 30720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4005}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'7554", 'reported_seq': 7678, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:10.878867+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:10.878867+0000', 'last_peered': '2024-09-11T05:16:10.878867+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:10.878867+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 7554, 'log_dups_size': 0, 'ondisk_log_size': 7554, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30924800, 'num_objects': 7550, 'num_object_clones': 0, 'num_object_copies': 15100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7550, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5957, 'num_write_kb': 23828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 4056}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'7640", 'reported_seq': 8211, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.476795+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:09.476795+0000', 'last_peered': '2024-09-11T05:16:09.476795+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:09.476795+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 7640, 'log_dups_size': 0, 'ondisk_log_size': 7640, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31272960, 'num_objects': 7635, 'num_object_clones': 0, 'num_object_copies': 15270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3386, 'num_objects_unfound': 0, 'num_objects_dirty': 7635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4184, 'num_write_kb': 16736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 212, 'num_bytes_recovered': 868352, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4247}, {'shards': '0,2', 'objects': 3386}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:14.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'7642", 'reported_seq': 7774, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.348529+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:09.348529+0000', 'last_peered': '2024-09-11T05:16:09.348529+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:09.348529+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 7642, 'log_dups_size': 0, 'ondisk_log_size': 7642, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31277056, 'num_objects': 7636, 'num_object_clones': 0, 'num_object_copies': 15272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4807, 'num_write_kb': 19228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4060}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'7642", 'reported_seq': 7773, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:10.891942+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:10.891942+0000', 'last_peered': '2024-09-11T05:16:10.891942+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:10.891942+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 7642, 'log_dups_size': 0, 'ondisk_log_size': 7642, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31272960, 'num_objects': 7635, 'num_object_clones': 0, 'num_object_copies': 15270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5710, 'num_write_kb': 22840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4110}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'7414", 'reported_seq': 7579, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.490718+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:09.490718+0000', 'last_peered': '2024-09-11T05:16:09.490718+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:09.490718+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 7414, 'log_dups_size': 0, 'ondisk_log_size': 7414, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30334976, 'num_objects': 7406, 'num_object_clones': 0, 'num_object_copies': 14812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4167, 'num_write_kb': 16668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3964}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'7646", 'reported_seq': 7763, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:10.789592+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:10.789592+0000', 'last_peered': '2024-09-11T05:16:10.789592+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:10.789592+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 7646, 'log_dups_size': 0, 'ondisk_log_size': 7646, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31297536, 'num_objects': 7641, 'num_object_clones': 0, 'num_object_copies': 15282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7641, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5675, 'num_write_kb': 22700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 4119}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'7686", 'reported_seq': 7817, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.423558+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:09.423558+0000', 'last_peered': '2024-09-11T05:16:09.423558+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:09.423558+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 7686, 'log_dups_size': 0, 'ondisk_log_size': 7686, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31473664, 'num_objects': 7684, 'num_object_clones': 0, 'num_object_copies': 15368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7684, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4719, 'num_write_kb': 18876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4061}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:14.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7430", 'reported_seq': 7563, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.417764+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:09.417764+0000', 'last_peered': '2024-09-11T05:16:09.417764+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:09.417764+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 7430, 'log_dups_size': 0, 'ondisk_log_size': 7430, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30400512, 'num_objects': 7422, 'num_object_clones': 0, 'num_object_copies': 14844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7422, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4296, 'num_write_kb': 17184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3927}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:14.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'7499", 'reported_seq': 7633, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.436416+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:09.436416+0000', 'last_peered': '2024-09-11T05:16:09.436416+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:09.436416+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 7499, 'log_dups_size': 0, 'ondisk_log_size': 7499, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30699520, 'num_objects': 7495, 'num_object_clones': 0, 'num_object_copies': 14990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4257, 'num_write_kb': 17028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3951}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:14.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'7447", 'reported_seq': 7590, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:09.362678+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:09.362678+0000', 'last_peered': '2024-09-11T05:16:09.362678+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:09.362678+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 7447, 'log_dups_size': 0, 'ondisk_log_size': 7447, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30454377, 'num_objects': 7436, 'num_object_clones': 0, 'num_object_copies': 14872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4454, 'num_write_kb': 17816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3921}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:14.750 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:15.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 162 20 123718 123698 2.98139 1.83594 0.0595048 0.0261955 2024-09-11T05:16:15.914 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:15.914 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/245546 objects degraded (14.429%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":253,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122773,"data_bytes":502866556,"bytes_used":1649332224,"bytes_avail":288260960256,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":245546,"degraded_ratio":0.14429475536152087,"misplaced_objects":3311,"misplaced_total":245546,"misplaced_ratio":0.013484235133131879,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70638,"recovering_keys_per_sec":0,"num_objects_recovered":207,"num_bytes_recovered":847872,"num_keys_recovered":0,"write_bytes_sec":2242692,"read_op_per_sec":0,"write_op_per_sec":547},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:15.980 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:15.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:16.138 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 163 20 124203 124183 2.97472 1.89453 0.0703832 0.0262539 2024-09-11T05:16:16.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:16.869 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:16.920 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-11T05:16:15.416007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502866556,"num_objects":122773,"num_object_clones":0,"num_object_copies":245546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3311,"num_objects_unfound":0,"num_objects_dirty":122773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85918,"num_write_kb":343666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3769,"num_bytes_recovered":15425555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122855,"ondisk_log_size":122855,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1610676,"kb_used_data":993668,"kb_used_omap":12,"kb_used_meta":616947,"kb_avail":281504844,"statfs":{"total":289910292480,"available":288260960256,"internally_reserved":0,"allocated":1017516032,"data_stored":1016673736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":631753729},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,6,5,19,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26918912,"num_objects":6572,"num_object_clones":0,"num_object_copies":13144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-207,"num_objects_unfound":0,"num_objects_dirty":6572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6572,"num_write_kb":26288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":847872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002943"},"pg_stats":[{"pgid":"2.f","version":"45'7840","reported_seq":7918,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.525997+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:14.525997+0000","last_peered":"2024-09-11T05:16:14.525997+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:14.525997+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7840,"log_dups_size":0,"ondisk_log_size":7840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32112640,"num_objects":7840,"num_object_clones":0,"num_object_copies":15680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7840,"num_write_kb":31360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4165},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7642","reported_seq":7744,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.854936+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:10.854936+0000","last_peered":"2024-09-11T05:16:10.854936+0000","last_clean":"2024-09-11T05:16:10.854936+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:10.854936+0000","last_undegraded":"2024-09-11T05:16:10.854936+0000","last_fullsized":"2024-09-11T05:16:10.854936+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6040,"num_write_kb":24160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7554","reported_seq":7678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.878867+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:10.878867+0000","last_peered":"2024-09-11T05:16:10.878867+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:10.878867+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7554,"log_dups_size":0,"ondisk_log_size":7554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30924800,"num_objects":7550,"num_object_clones":0,"num_object_copies":15100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":23828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4056}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7659","reported_seq":7730,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.849194+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:10.849194+0000","last_peered":"2024-09-11T05:16:10.849194+0000","last_clean":"2024-09-11T05:16:10.849194+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:10.849194+0000","last_undegraded":"2024-09-11T05:16:10.849194+0000","last_fullsized":"2024-09-11T05:16:10.849194+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7659,"log_dups_size":0,"ondisk_log_size":7659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31371264,"num_objects":7659,"num_object_clones":0,"num_object_copies":15318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7659,"num_write_kb":30636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7832","reported_seq":7961,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:14.443772+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:14.443772+0000","last_peered":"2024-09-11T05:16:14.443772+0000","last_clean":"2024-09-11T05:16:14.443772+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:14.443772+0000","last_undegraded":"2024-09-11T05:16:14.443772+0000","last_fullsized":"2024-09-11T05:16:14.443772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7832,"log_dups_size":0,"ondisk_log_size":7832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32051200,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7794","reported_seq":8518,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.517160+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:14.517160+0000","last_peered":"2024-09-11T05:16:14.517160+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:14.517160+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7794,"log_dups_size":0,"ondisk_log_size":7794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31903744,"num_objects":7789,"num_object_clones":0,"num_object_copies":15578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3311,"num_objects_unfound":0,"num_objects_dirty":7789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4476},{"shards":"0,2","objects":3311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7801","reported_seq":7933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.507983+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:14.507983+0000","last_peered":"2024-09-11T05:16:14.507983+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:14.507983+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7801,"log_dups_size":0,"ondisk_log_size":7801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31928320,"num_objects":7795,"num_object_clones":0,"num_object_copies":15590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4219},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7677","reported_seq":7801,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:14.462633+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:14.462633+0000","last_peered":"2024-09-11T05:16:14.462633+0000","last_clean":"2024-09-11T05:16:14.462633+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:14.462633+0000","last_undegraded":"2024-09-11T05:16:14.462633+0000","last_fullsized":"2024-09-11T05:16:14.462633+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7677,"log_dups_size":0,"ondisk_log_size":7677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31436800,"num_objects":7675,"num_object_clones":0,"num_object_copies":15350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4980,"num_write_kb":19920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7642","reported_seq":7773,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.891942+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:10.891942+0000","last_peered":"2024-09-11T05:16:10.891942+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:10.891942+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7642,"log_dups_size":0,"ondisk_log_size":7642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31272960,"num_objects":7635,"num_object_clones":0,"num_object_copies":15270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5710,"num_write_kb":22840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7571","reported_seq":7735,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.521323+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:14.521323+0000","last_peered":"2024-09-11T05:16:14.521323+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:14.521323+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7571,"log_dups_size":0,"ondisk_log_size":7571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30978048,"num_objects":7563,"num_object_clones":0,"num_object_copies":15126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4324,"num_write_kb":17296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4121},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7646","reported_seq":7763,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:10.789592+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:10.789592+0000","last_peered":"2024-09-11T05:16:10.789592+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:10.789592+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7646,"log_dups_size":0,"ondisk_log_size":7646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31297536,"num_objects":7641,"num_object_clones":0,"num_object_copies":15282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":78,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570072+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570072+0000","last_peered":"2024-09-11T05:15:57.570072+0000","last_clean":"2024-09-11T05:15:57.570072+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570072+0000","last_undegraded":"2024-09-11T05:15:57.570072+0000","last_fullsized":"2024-09-11T05:15:57.570072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7494","reported_seq":14579,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:10.906192+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:10.906192+0000","last_peered":"2024-09-11T05:16:10.906192+0000","last_clean":"2024-09-11T05:16:10.906192+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:10.906192+0000","last_undegraded":"2024-09-11T05:16:10.906192+0000","last_fullsized":"2024-09-11T05:16:10.906192+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7494,"log_dups_size":0,"ondisk_log_size":7494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30670848,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":21588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7845","reported_seq":7975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.546646+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:14.546646+0000","last_peered":"2024-09-11T05:16:14.546646+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:14.546646+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7845,"log_dups_size":0,"ondisk_log_size":7845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4878,"num_write_kb":19512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4220},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570756+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570756+0000","last_peered":"2024-09-11T05:15:57.570756+0000","last_clean":"2024-09-11T05:15:57.570756+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570756+0000","last_undegraded":"2024-09-11T05:15:57.570756+0000","last_fullsized":"2024-09-11T05:15:57.570756+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7576","reported_seq":7709,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.547937+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:14.547937+0000","last_peered":"2024-09-11T05:16:14.547937+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:14.547937+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7576,"log_dups_size":0,"ondisk_log_size":7576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":7568,"num_object_clones":0,"num_object_copies":15136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4073},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7681","reported_seq":7816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.463618+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:14.463618+0000","last_peered":"2024-09-11T05:16:14.463618+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:14.463618+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7681,"log_dups_size":0,"ondisk_log_size":7681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4439,"num_write_kb":17756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570579+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570579+0000","last_peered":"2024-09-11T05:15:57.570579+0000","last_clean":"2024-09-11T05:15:57.570579+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570579+0000","last_undegraded":"2024-09-11T05:15:57.570579+0000","last_fullsized":"2024-09-11T05:15:57.570579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7598","reported_seq":7741,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.536335+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:14.536335+0000","last_peered":"2024-09-11T05:16:14.536335+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:14.536335+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7598,"log_dups_size":0,"ondisk_log_size":7598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31072873,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4605,"num_write_kb":18420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4072},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502866537,"num_objects":122771,"num_object_clones":0,"num_object_copies":245542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3311,"num_objects_unfound":0,"num_objects_dirty":122771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85916,"num_write_kb":343664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3768,"num_bytes_recovered":15425536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1018621952,"data_stored":1018541935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122852,"ondisk_log_size":122852,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537016,"kb_used_data":356776,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93834824,"statfs":{"total":96636764160,"available":96086859776,"internally_reserved":0,"allocated":365338624,"data_stored":365083770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184544939},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376504,"kb_used_data":210152,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93995336,"statfs":{"total":96636764160,"available":96251224064,"internally_reserved":0,"allocated":215195648,"data_stored":214867137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170323627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215149,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697156,"kb_used_data":426740,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93674684,"statfs":{"total":96636764160,"available":95922876416,"internally_reserved":0,"allocated":436981760,"data_stored":436722829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,4,1,8,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437125120,"data_stored":437121641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215560192,"data_stored":215487133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365936640,"data_stored":365933161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:17.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 164 20 124689 124669 2.96816 1.89844 0.0218314 0.0263132 2024-09-11T05:16:18.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 165 20 125190 125170 2.96203 1.95703 0.0206949 0.0263674 2024-09-11T05:16:19.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 166 20 125695 125675 2.95607 1.97266 0.0560394 0.0264197 2024-09-11T05:16:19.923 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:20.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 167 20 126198 126178 2.95014 1.96484 0.0482188 0.0264742 2024-09-11T05:16:20.828 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:20.830 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:20.887 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-11T05:16:19.417087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506565244,"num_objects":123676,"num_object_clones":0,"num_object_copies":247352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3311,"num_objects_unfound":0,"num_objects_dirty":123676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86821,"num_write_kb":347278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3769,"num_bytes_recovered":15425555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123758,"ondisk_log_size":123758,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1632880,"kb_used_data":1006144,"kb_used_omap":12,"kb_used_meta":626675,"kb_avail":281482640,"statfs":{"total":289910292480,"available":288238223360,"internally_reserved":0,"allocated":1030291456,"data_stored":1029449160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":641715201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,4,8,5,10,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20275200,"num_objects":4950,"num_object_clones":0,"num_object_copies":9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-143,"num_objects_unfound":0,"num_objects_dirty":4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":19800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":585728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003168"},"pg_stats":[{"pgid":"2.f","version":"45'7840","reported_seq":7918,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.525997+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:14.525997+0000","last_peered":"2024-09-11T05:16:14.525997+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:14.525997+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7840,"log_dups_size":0,"ondisk_log_size":7840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32112640,"num_objects":7840,"num_object_clones":0,"num_object_copies":15680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7840,"num_write_kb":31360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4165},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7787","reported_seq":7889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:15.856435+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:15.856435+0000","last_peered":"2024-09-11T05:16:15.856435+0000","last_clean":"2024-09-11T05:16:15.856435+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:15.856435+0000","last_undegraded":"2024-09-11T05:16:15.856435+0000","last_fullsized":"2024-09-11T05:16:15.856435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7787,"log_dups_size":0,"ondisk_log_size":7787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31870976,"num_objects":7781,"num_object_clones":0,"num_object_copies":15562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6185,"num_write_kb":24740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7693","reported_seq":7818,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:15.886932+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:15.886932+0000","last_peered":"2024-09-11T05:16:15.886932+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:15.886932+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7693,"log_dups_size":0,"ondisk_log_size":7693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6096,"num_write_kb":24384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4195}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7814","reported_seq":7884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:15.735568+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:15.735568+0000","last_peered":"2024-09-11T05:16:15.735568+0000","last_clean":"2024-09-11T05:16:15.735568+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:15.735568+0000","last_undegraded":"2024-09-11T05:16:15.735568+0000","last_fullsized":"2024-09-11T05:16:15.735568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7814,"log_dups_size":0,"ondisk_log_size":7814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":31256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7832","reported_seq":7961,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:14.443772+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:14.443772+0000","last_peered":"2024-09-11T05:16:14.443772+0000","last_clean":"2024-09-11T05:16:14.443772+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:14.443772+0000","last_undegraded":"2024-09-11T05:16:14.443772+0000","last_fullsized":"2024-09-11T05:16:14.443772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":7832,"log_dups_size":0,"ondisk_log_size":7832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32051200,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7794","reported_seq":8518,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.517160+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:14.517160+0000","last_peered":"2024-09-11T05:16:14.517160+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:14.517160+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7794,"log_dups_size":0,"ondisk_log_size":7794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31903744,"num_objects":7789,"num_object_clones":0,"num_object_copies":15578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3311,"num_objects_unfound":0,"num_objects_dirty":7789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4476},{"shards":"0,2","objects":3311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7801","reported_seq":7933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.507983+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:14.507983+0000","last_peered":"2024-09-11T05:16:14.507983+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:14.507983+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7801,"log_dups_size":0,"ondisk_log_size":7801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31928320,"num_objects":7795,"num_object_clones":0,"num_object_copies":15590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4219},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7677","reported_seq":7801,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:14.462633+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:14.462633+0000","last_peered":"2024-09-11T05:16:14.462633+0000","last_clean":"2024-09-11T05:16:14.462633+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:14.462633+0000","last_undegraded":"2024-09-11T05:16:14.462633+0000","last_fullsized":"2024-09-11T05:16:14.462633+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7677,"log_dups_size":0,"ondisk_log_size":7677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31436800,"num_objects":7675,"num_object_clones":0,"num_object_copies":15350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4980,"num_write_kb":19920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7800","reported_seq":7931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:15.871636+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:15.871636+0000","last_peered":"2024-09-11T05:16:15.871636+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:15.871636+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7800,"log_dups_size":0,"ondisk_log_size":7800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31920128,"num_objects":7793,"num_object_clones":0,"num_object_copies":15586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5868,"num_write_kb":23472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4268}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7571","reported_seq":7735,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.521323+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:14.521323+0000","last_peered":"2024-09-11T05:16:14.521323+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:14.521323+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7571,"log_dups_size":0,"ondisk_log_size":7571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30978048,"num_objects":7563,"num_object_clones":0,"num_object_copies":15126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4324,"num_write_kb":17296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4121},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7794","reported_seq":7911,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:15.851454+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:15.851454+0000","last_peered":"2024-09-11T05:16:15.851454+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:15.851454+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7794,"log_dups_size":0,"ondisk_log_size":7794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31903744,"num_objects":7789,"num_object_clones":0,"num_object_copies":15578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7652","reported_seq":14737,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:15.862906+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:15.862906+0000","last_peered":"2024-09-11T05:16:15.862906+0000","last_clean":"2024-09-11T05:16:15.862906+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:15.862906+0000","last_undegraded":"2024-09-11T05:16:15.862906+0000","last_fullsized":"2024-09-11T05:16:15.862906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7652,"log_dups_size":0,"ondisk_log_size":7652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31318016,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5555,"num_write_kb":22220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7845","reported_seq":7975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.546646+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:14.546646+0000","last_peered":"2024-09-11T05:16:14.546646+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:14.546646+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7845,"log_dups_size":0,"ondisk_log_size":7845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4878,"num_write_kb":19512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4220},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7576","reported_seq":7709,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.547937+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:14.547937+0000","last_peered":"2024-09-11T05:16:14.547937+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:14.547937+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7576,"log_dups_size":0,"ondisk_log_size":7576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":7568,"num_object_clones":0,"num_object_copies":15136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4073},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7681","reported_seq":7816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.463618+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:14.463618+0000","last_peered":"2024-09-11T05:16:14.463618+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:14.463618+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7681,"log_dups_size":0,"ondisk_log_size":7681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4439,"num_write_kb":17756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7598","reported_seq":7741,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:14.536335+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:14.536335+0000","last_peered":"2024-09-11T05:16:14.536335+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:14.536335+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7598,"log_dups_size":0,"ondisk_log_size":7598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31072873,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4605,"num_write_kb":18420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4072},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506565225,"num_objects":123674,"num_object_clones":0,"num_object_copies":247348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3311,"num_objects_unfound":0,"num_objects_dirty":123674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86819,"num_write_kb":347276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3768,"num_bytes_recovered":15425536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1031544832,"data_stored":1031464815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123755,"ondisk_log_size":123755,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546128,"kb_used_data":361728,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93825712,"statfs":{"total":96636764160,"available":96077529088,"internally_reserved":0,"allocated":370409472,"data_stored":370154618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188804779},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,0,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389596,"kb_used_data":217676,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93982244,"statfs":{"total":96636764160,"available":96237817856,"internally_reserved":0,"allocated":222900224,"data_stored":222571713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215149,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697156,"kb_used_data":426740,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93674684,"statfs":{"total":96636764160,"available":95922876416,"internally_reserved":0,"allocated":436981760,"data_stored":436722829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,4,1,8,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437125120,"data_stored":437121641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223477760,"data_stored":223404701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370941952,"data_stored":370938473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:20.889 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:21.137 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 168 20 126670 126650 2.94355 1.84375 0.0380257 0.0265337 2024-09-11T05:16:21.598 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:21.678 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-11T05:16:21.417634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516751996,"num_objects":126163,"num_object_clones":0,"num_object_copies":252326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3249,"num_objects_unfound":0,"num_objects_dirty":126163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89308,"num_write_kb":357226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":15679507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126245,"ondisk_log_size":126245,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1678632,"kb_used_data":1018552,"kb_used_omap":12,"kb_used_meta":660019,"kb_avail":281436888,"statfs":{"total":289910292480,"available":288191373312,"internally_reserved":0,"allocated":1042997248,"data_stored":1042154952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":675859457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,6,7,2,14,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-205,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7437,"num_write_kb":29748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":839680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003184"},"pg_stats":[{"pgid":"2.f","version":"45'7985","reported_seq":8063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.593053+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:19.593053+0000","last_peered":"2024-09-11T05:16:19.593053+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:19.593053+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7985,"log_dups_size":0,"ondisk_log_size":7985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32706560,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7985,"num_write_kb":31940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4310},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7958","reported_seq":8060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.904160+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:20.904160+0000","last_peered":"2024-09-11T05:16:20.904160+0000","last_clean":"2024-09-11T05:16:20.904160+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:20.904160+0000","last_undegraded":"2024-09-11T05:16:20.904160+0000","last_fullsized":"2024-09-11T05:16:20.904160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7958,"log_dups_size":0,"ondisk_log_size":7958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6356,"num_write_kb":25424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7846","reported_seq":7971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.771828+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:20.771828+0000","last_peered":"2024-09-11T05:16:20.771828+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:20.771828+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7846,"log_dups_size":0,"ondisk_log_size":7846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32120832,"num_objects":7842,"num_object_clones":0,"num_object_copies":15684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6249,"num_write_kb":24996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4348}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7961","reported_seq":8032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.822576+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:20.822576+0000","last_peered":"2024-09-11T05:16:20.822576+0000","last_clean":"2024-09-11T05:16:20.822576+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:20.822576+0000","last_undegraded":"2024-09-11T05:16:20.822576+0000","last_fullsized":"2024-09-11T05:16:20.822576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7961,"log_dups_size":0,"ondisk_log_size":7961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7961,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8002","reported_seq":8131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:19.581376+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:19.581376+0000","last_peered":"2024-09-11T05:16:19.581376+0000","last_clean":"2024-09-11T05:16:19.581376+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:19.581376+0000","last_undegraded":"2024-09-11T05:16:19.581376+0000","last_fullsized":"2024-09-11T05:16:19.581376+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8002,"log_dups_size":0,"ondisk_log_size":8002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32747520,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4836,"num_write_kb":19344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7951","reported_seq":8797,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.567104+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:19.567104+0000","last_peered":"2024-09-11T05:16:19.567104+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:19.567104+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7951,"log_dups_size":0,"ondisk_log_size":7951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3249,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4495,"num_write_kb":17980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":1429504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4695},{"shards":"0,2","objects":3249}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7940","reported_seq":8072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.481574+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:19.481574+0000","last_peered":"2024-09-11T05:16:19.481574+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:19.481574+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7940,"log_dups_size":0,"ondisk_log_size":7940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32497664,"num_objects":7934,"num_object_clones":0,"num_object_copies":15868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5105,"num_write_kb":20420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4358},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7835","reported_seq":7959,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:19.597382+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:19.597382+0000","last_peered":"2024-09-11T05:16:19.597382+0000","last_clean":"2024-09-11T05:16:19.597382+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:19.597382+0000","last_undegraded":"2024-09-11T05:16:19.597382+0000","last_fullsized":"2024-09-11T05:16:19.597382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7835,"log_dups_size":0,"ondisk_log_size":7835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32083968,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5138,"num_write_kb":20552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7968","reported_seq":8097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.875143+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:20.875143+0000","last_peered":"2024-09-11T05:16:20.875143+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:20.875143+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7968,"log_dups_size":0,"ondisk_log_size":7968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6036,"num_write_kb":24144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7726","reported_seq":7888,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.546927+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:19.546927+0000","last_peered":"2024-09-11T05:16:19.546927+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:19.546927+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4479,"num_write_kb":17916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4276},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7955","reported_seq":8072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.841799+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:20.841799+0000","last_peered":"2024-09-11T05:16:20.841799+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:20.841799+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7955,"log_dups_size":0,"ondisk_log_size":7955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5984,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7813","reported_seq":14898,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.848176+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:20.848176+0000","last_peered":"2024-09-11T05:16:20.848176+0000","last_clean":"2024-09-11T05:16:20.848176+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:20.848176+0000","last_undegraded":"2024-09-11T05:16:20.848176+0000","last_fullsized":"2024-09-11T05:16:20.848176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7813,"log_dups_size":0,"ondisk_log_size":7813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5716,"num_write_kb":22864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7991","reported_seq":8122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.517961+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:19.517961+0000","last_peered":"2024-09-11T05:16:19.517961+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:19.517961+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7991,"log_dups_size":0,"ondisk_log_size":7991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722944,"num_objects":7989,"num_object_clones":0,"num_object_copies":15978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5024,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4366},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7743","reported_seq":7876,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.600581+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:19.600581+0000","last_peered":"2024-09-11T05:16:19.600581+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:19.600581+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7743,"log_dups_size":0,"ondisk_log_size":7743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31682560,"num_objects":7735,"num_object_clones":0,"num_object_copies":15470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4609,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4240},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7821","reported_seq":7956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.553850+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:19.553850+0000","last_peered":"2024-09-11T05:16:19.553850+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:19.553850+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7821,"log_dups_size":0,"ondisk_log_size":7821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018432,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4579,"num_write_kb":18316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4273},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7747","reported_seq":7890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.597820+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:19.597820+0000","last_peered":"2024-09-11T05:16:19.597820+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:19.597820+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7747,"log_dups_size":0,"ondisk_log_size":7747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31683177,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4221},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516751977,"num_objects":126161,"num_object_clones":0,"num_object_copies":252322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3249,"num_objects_unfound":0,"num_objects_dirty":126161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89306,"num_write_kb":357224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3830,"num_bytes_recovered":15679488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1044365312,"data_stored":1044285295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126242,"ondisk_log_size":126242,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592948,"kb_used_data":366884,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93778892,"statfs":{"total":96636764160,"available":96029585408,"internally_reserved":0,"allocated":375689216,"data_stored":375434362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389596,"kb_used_data":217676,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93982244,"statfs":{"total":96636764160,"available":96237817856,"internally_reserved":0,"allocated":222900224,"data_stored":222571713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696088,"kb_used_data":433992,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93675752,"statfs":{"total":96636764160,"available":95923970048,"internally_reserved":0,"allocated":444407808,"data_stored":444148877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268365483},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,0,1,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444637184,"data_stored":444633705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223477760,"data_stored":223404701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376250368,"data_stored":376246889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:21.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:21.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'7985", 'reported_seq': 8063, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.593053+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:19.593053+0000', 'last_peered': '2024-09-11T05:16:19.593053+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:19.593053+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 7985, 'log_dups_size': 0, 'ondisk_log_size': 7985, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32706560, 'num_objects': 7985, 'num_object_clones': 0, 'num_object_copies': 15970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7985, 'num_write_kb': 31940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4310}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'7846", 'reported_seq': 7971, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:20.771828+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:20.771828+0000', 'last_peered': '2024-09-11T05:16:20.771828+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:20.771828+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 7846, 'log_dups_size': 0, 'ondisk_log_size': 7846, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32120832, 'num_objects': 7842, 'num_object_clones': 0, 'num_object_copies': 15684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6249, 'num_write_kb': 24996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 4348}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'7951", 'reported_seq': 8797, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.567104+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:19.567104+0000', 'last_peered': '2024-09-11T05:16:19.567104+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:19.567104+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 7951, 'log_dups_size': 0, 'ondisk_log_size': 7951, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32546816, 'num_objects': 7946, 'num_object_clones': 0, 'num_object_copies': 15892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3249, 'num_objects_unfound': 0, 'num_objects_dirty': 7946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4495, 'num_write_kb': 17980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 349, 'num_bytes_recovered': 1429504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4695}, {'shards': '0,2', 'objects': 3249}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'7940", 'reported_seq': 8072, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.481574+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:19.481574+0000', 'last_peered': '2024-09-11T05:16:19.481574+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:19.481574+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 7940, 'log_dups_size': 0, 'ondisk_log_size': 7940, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32497664, 'num_objects': 7934, 'num_object_clones': 0, 'num_object_copies': 15868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7934, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5105, 'num_write_kb': 20420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4358}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'7968", 'reported_seq': 8097, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:20.875143+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:20.875143+0000', 'last_peered': '2024-09-11T05:16:20.875143+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:20.875143+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 7968, 'log_dups_size': 0, 'ondisk_log_size': 7968, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32608256, 'num_objects': 7961, 'num_object_clones': 0, 'num_object_copies': 15922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6036, 'num_write_kb': 24144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:21.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'7726", 'reported_seq': 7888, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.546927+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:19.546927+0000', 'last_peered': '2024-09-11T05:16:19.546927+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:19.546927+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 7726, 'log_dups_size': 0, 'ondisk_log_size': 7726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31612928, 'num_objects': 7718, 'num_object_clones': 0, 'num_object_copies': 15436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4479, 'num_write_kb': 17916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4276}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'7955", 'reported_seq': 8072, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:20.841799+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:20.841799+0000', 'last_peered': '2024-09-11T05:16:20.841799+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:20.841799+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 7955, 'log_dups_size': 0, 'ondisk_log_size': 7955, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32563200, 'num_objects': 7950, 'num_object_clones': 0, 'num_object_copies': 15900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5984, 'num_write_kb': 23936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 4428}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'7991", 'reported_seq': 8122, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.517961+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:19.517961+0000', 'last_peered': '2024-09-11T05:16:19.517961+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:19.517961+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 7991, 'log_dups_size': 0, 'ondisk_log_size': 7991, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32722944, 'num_objects': 7989, 'num_object_clones': 0, 'num_object_copies': 15978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5024, 'num_write_kb': 20096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4366}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7743", 'reported_seq': 7876, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.600581+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:19.600581+0000', 'last_peered': '2024-09-11T05:16:19.600581+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:19.600581+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 7743, 'log_dups_size': 0, 'ondisk_log_size': 7743, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31682560, 'num_objects': 7735, 'num_object_clones': 0, 'num_object_copies': 15470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7735, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4609, 'num_write_kb': 18436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4240}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'7821", 'reported_seq': 7956, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.553850+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:19.553850+0000', 'last_peered': '2024-09-11T05:16:19.553850+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:19.553850+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 7821, 'log_dups_size': 0, 'ondisk_log_size': 7821, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32018432, 'num_objects': 7817, 'num_object_clones': 0, 'num_object_copies': 15634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4579, 'num_write_kb': 18316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4273}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:21.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'7747", 'reported_seq': 7890, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:19.597820+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:19.597820+0000', 'last_peered': '2024-09-11T05:16:19.597820+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:19.597820+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 7747, 'log_dups_size': 0, 'ondisk_log_size': 7747, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31683177, 'num_objects': 7736, 'num_object_clones': 0, 'num_object_copies': 15472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4754, 'num_write_kb': 19016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4221}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:21.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:22.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 169 20 127111 127091 2.93633 1.72266 0.018262 0.0265979 2024-09-11T05:16:22.604 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:22.604 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/252326 objects degraded (14.042%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":260,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":126163,"data_bytes":516751996,"bytes_used":1718919168,"bytes_avail":288191373312,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":252326,"degraded_ratio":0.14041755506765058,"misplaced_objects":3249,"misplaced_total":252326,"misplaced_ratio":0.012876199836719164,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":69954,"recovering_keys_per_sec":0,"num_objects_recovered":205,"num_bytes_recovered":839680,"num_keys_recovered":0,"write_bytes_sec":2537822,"read_op_per_sec":0,"write_op_per_sec":619},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:22.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:22.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:23.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 170 20 127570 127550 2.9296 1.79297 0.014782 0.026657 2024-09-11T05:16:23.374 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:23.375 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:23.427 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-11T05:16:21.417634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516751996,"num_objects":126163,"num_object_clones":0,"num_object_copies":252326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3249,"num_objects_unfound":0,"num_objects_dirty":126163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89308,"num_write_kb":357226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":15679507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126245,"ondisk_log_size":126245,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1678632,"kb_used_data":1018552,"kb_used_omap":12,"kb_used_meta":660019,"kb_avail":281436888,"statfs":{"total":289910292480,"available":288191373312,"internally_reserved":0,"allocated":1042997248,"data_stored":1042154952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":675859457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,6,7,2,14,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-205,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7437,"num_write_kb":29748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":839680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003184"},"pg_stats":[{"pgid":"2.f","version":"45'7985","reported_seq":8063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.593053+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:19.593053+0000","last_peered":"2024-09-11T05:16:19.593053+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:19.593053+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7985,"log_dups_size":0,"ondisk_log_size":7985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32706560,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7985,"num_write_kb":31940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4310},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7958","reported_seq":8060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.904160+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:20.904160+0000","last_peered":"2024-09-11T05:16:20.904160+0000","last_clean":"2024-09-11T05:16:20.904160+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:20.904160+0000","last_undegraded":"2024-09-11T05:16:20.904160+0000","last_fullsized":"2024-09-11T05:16:20.904160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7958,"log_dups_size":0,"ondisk_log_size":7958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6356,"num_write_kb":25424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7846","reported_seq":7971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.771828+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:20.771828+0000","last_peered":"2024-09-11T05:16:20.771828+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:20.771828+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7846,"log_dups_size":0,"ondisk_log_size":7846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32120832,"num_objects":7842,"num_object_clones":0,"num_object_copies":15684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6249,"num_write_kb":24996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4348}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7961","reported_seq":8032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.822576+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:20.822576+0000","last_peered":"2024-09-11T05:16:20.822576+0000","last_clean":"2024-09-11T05:16:20.822576+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:20.822576+0000","last_undegraded":"2024-09-11T05:16:20.822576+0000","last_fullsized":"2024-09-11T05:16:20.822576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7961,"log_dups_size":0,"ondisk_log_size":7961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7961,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8002","reported_seq":8131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:19.581376+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:19.581376+0000","last_peered":"2024-09-11T05:16:19.581376+0000","last_clean":"2024-09-11T05:16:19.581376+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:19.581376+0000","last_undegraded":"2024-09-11T05:16:19.581376+0000","last_fullsized":"2024-09-11T05:16:19.581376+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8002,"log_dups_size":0,"ondisk_log_size":8002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32747520,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4836,"num_write_kb":19344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7951","reported_seq":8797,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.567104+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:19.567104+0000","last_peered":"2024-09-11T05:16:19.567104+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:19.567104+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":7951,"log_dups_size":0,"ondisk_log_size":7951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3249,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4495,"num_write_kb":17980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":1429504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4695},{"shards":"0,2","objects":3249}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7940","reported_seq":8072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.481574+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:19.481574+0000","last_peered":"2024-09-11T05:16:19.481574+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:19.481574+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":7940,"log_dups_size":0,"ondisk_log_size":7940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32497664,"num_objects":7934,"num_object_clones":0,"num_object_copies":15868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5105,"num_write_kb":20420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4358},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7835","reported_seq":7959,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:19.597382+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:19.597382+0000","last_peered":"2024-09-11T05:16:19.597382+0000","last_clean":"2024-09-11T05:16:19.597382+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:19.597382+0000","last_undegraded":"2024-09-11T05:16:19.597382+0000","last_fullsized":"2024-09-11T05:16:19.597382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":7835,"log_dups_size":0,"ondisk_log_size":7835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32083968,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5138,"num_write_kb":20552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7968","reported_seq":8097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.875143+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:20.875143+0000","last_peered":"2024-09-11T05:16:20.875143+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:20.875143+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7968,"log_dups_size":0,"ondisk_log_size":7968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6036,"num_write_kb":24144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7726","reported_seq":7888,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.546927+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:19.546927+0000","last_peered":"2024-09-11T05:16:19.546927+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:19.546927+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4479,"num_write_kb":17916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4276},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7955","reported_seq":8072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.841799+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:20.841799+0000","last_peered":"2024-09-11T05:16:20.841799+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:20.841799+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7955,"log_dups_size":0,"ondisk_log_size":7955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5984,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7813","reported_seq":14898,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.848176+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:20.848176+0000","last_peered":"2024-09-11T05:16:20.848176+0000","last_clean":"2024-09-11T05:16:20.848176+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:20.848176+0000","last_undegraded":"2024-09-11T05:16:20.848176+0000","last_fullsized":"2024-09-11T05:16:20.848176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7813,"log_dups_size":0,"ondisk_log_size":7813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5716,"num_write_kb":22864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7991","reported_seq":8122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.517961+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:19.517961+0000","last_peered":"2024-09-11T05:16:19.517961+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:19.517961+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":7991,"log_dups_size":0,"ondisk_log_size":7991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32722944,"num_objects":7989,"num_object_clones":0,"num_object_copies":15978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5024,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4366},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7743","reported_seq":7876,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.600581+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:19.600581+0000","last_peered":"2024-09-11T05:16:19.600581+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:19.600581+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7743,"log_dups_size":0,"ondisk_log_size":7743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31682560,"num_objects":7735,"num_object_clones":0,"num_object_copies":15470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4609,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4240},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7821","reported_seq":7956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.553850+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:19.553850+0000","last_peered":"2024-09-11T05:16:19.553850+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:19.553850+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7821,"log_dups_size":0,"ondisk_log_size":7821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018432,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4579,"num_write_kb":18316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4273},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570108+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570108+0000","last_peered":"2024-09-11T05:15:57.570108+0000","last_clean":"2024-09-11T05:15:57.570108+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570108+0000","last_undegraded":"2024-09-11T05:15:57.570108+0000","last_fullsized":"2024-09-11T05:15:57.570108+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7747","reported_seq":7890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:19.597820+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:19.597820+0000","last_peered":"2024-09-11T05:16:19.597820+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:19.597820+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7747,"log_dups_size":0,"ondisk_log_size":7747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31683177,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4221},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516751977,"num_objects":126161,"num_object_clones":0,"num_object_copies":252322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3249,"num_objects_unfound":0,"num_objects_dirty":126161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89306,"num_write_kb":357224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3830,"num_bytes_recovered":15679488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1044365312,"data_stored":1044285295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126242,"ondisk_log_size":126242,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592948,"kb_used_data":366884,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93778892,"statfs":{"total":96636764160,"available":96029585408,"internally_reserved":0,"allocated":375689216,"data_stored":375434362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389596,"kb_used_data":217676,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93982244,"statfs":{"total":96636764160,"available":96237817856,"internally_reserved":0,"allocated":222900224,"data_stored":222571713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":176025259},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696088,"kb_used_data":433992,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93675752,"statfs":{"total":96636764160,"available":95923970048,"internally_reserved":0,"allocated":444407808,"data_stored":444148877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268365483},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,0,1,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444637184,"data_stored":444633705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223477760,"data_stored":223404701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376250368,"data_stored":376246889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:24.141 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 171 20 128061 128041 2.92369 1.91797 0.0669705 0.0267138 2024-09-11T05:16:25.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 172 20 128570 128550 2.91825 1.98828 0.0135428 0.0267617 2024-09-11T05:16:26.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 173 20 129069 129049 2.91265 1.94922 0.00953603 0.0268151 2024-09-11T05:16:26.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:27.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 174 20 129546 129526 2.90662 1.86328 0.0565053 0.0268704 2024-09-11T05:16:27.309 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:27.311 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:27.342 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-11T05:16:25.420212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":522953340,"num_objects":127677,"num_object_clones":0,"num_object_copies":255354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":127677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90822,"num_write_kb":363282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":15953939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127759,"ondisk_log_size":127759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1705908,"kb_used_data":1033348,"kb_used_omap":12,"kb_used_meta":672499,"kb_avail":281409612,"statfs":{"total":289910292480,"available":288163442688,"internally_reserved":0,"allocated":1058148352,"data_stored":1057306056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":688638977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,0,6,4,13,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26628096,"num_objects":6501,"num_object_clones":0,"num_object_copies":13002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-204,"num_objects_unfound":0,"num_objects_dirty":6501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6501,"num_write_kb":26004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004751"},"pg_stats":[{"pgid":"2.f","version":"45'8128","reported_seq":8206,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.638467+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:24.638467+0000","last_peered":"2024-09-11T05:16:24.638467+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:24.638467+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8128,"log_dups_size":0,"ondisk_log_size":8128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33292288,"num_objects":8128,"num_object_clones":0,"num_object_copies":16256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8128,"num_write_kb":32512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7958","reported_seq":8060,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.904160+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:20.904160+0000","last_peered":"2024-09-11T05:16:20.904160+0000","last_clean":"2024-09-11T05:16:20.904160+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:20.904160+0000","last_undegraded":"2024-09-11T05:16:20.904160+0000","last_fullsized":"2024-09-11T05:16:20.904160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":7958,"log_dups_size":0,"ondisk_log_size":7958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6356,"num_write_kb":25424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7846","reported_seq":7971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.771828+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:20.771828+0000","last_peered":"2024-09-11T05:16:20.771828+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:20.771828+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7846,"log_dups_size":0,"ondisk_log_size":7846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32120832,"num_objects":7842,"num_object_clones":0,"num_object_copies":15684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6249,"num_write_kb":24996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4348}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7961","reported_seq":8032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.822576+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:20.822576+0000","last_peered":"2024-09-11T05:16:20.822576+0000","last_clean":"2024-09-11T05:16:20.822576+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:20.822576+0000","last_undegraded":"2024-09-11T05:16:20.822576+0000","last_fullsized":"2024-09-11T05:16:20.822576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":7961,"log_dups_size":0,"ondisk_log_size":7961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7961,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8155","reported_seq":8284,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:24.593049+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:24.593049+0000","last_peered":"2024-09-11T05:16:24.593049+0000","last_clean":"2024-09-11T05:16:24.593049+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:24.593049+0000","last_undegraded":"2024-09-11T05:16:24.593049+0000","last_fullsized":"2024-09-11T05:16:24.593049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33374208,"num_objects":8148,"num_object_clones":0,"num_object_copies":16296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8087","reported_seq":9068,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.647320+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:24.647320+0000","last_peered":"2024-09-11T05:16:24.647320+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:24.647320+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8087,"log_dups_size":0,"ondisk_log_size":8087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4631,"num_write_kb":18524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4898},{"shards":"0,2","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8066","reported_seq":8198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.604488+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:24.604488+0000","last_peered":"2024-09-11T05:16:24.604488+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:24.604488+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33013760,"num_objects":8060,"num_object_clones":0,"num_object_copies":16120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5231,"num_write_kb":20924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8004","reported_seq":8128,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:24.603807+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:24.603807+0000","last_peered":"2024-09-11T05:16:24.603807+0000","last_clean":"2024-09-11T05:16:24.603807+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:24.603807+0000","last_undegraded":"2024-09-11T05:16:24.603807+0000","last_fullsized":"2024-09-11T05:16:24.603807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8004,"log_dups_size":0,"ondisk_log_size":8004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32776192,"num_objects":8002,"num_object_clones":0,"num_object_copies":16004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7968","reported_seq":8097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.875143+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:20.875143+0000","last_peered":"2024-09-11T05:16:20.875143+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:20.875143+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":7968,"log_dups_size":0,"ondisk_log_size":7968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6036,"num_write_kb":24144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4436}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7874","reported_seq":8037,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.614517+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:24.614517+0000","last_peered":"2024-09-11T05:16:24.614517+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:24.614517+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7874,"log_dups_size":0,"ondisk_log_size":7874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4424},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7955","reported_seq":8072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:20.841799+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:20.841799+0000","last_peered":"2024-09-11T05:16:20.841799+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:20.841799+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":7955,"log_dups_size":0,"ondisk_log_size":7955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5984,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7813","reported_seq":14898,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:20.848176+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:20.848176+0000","last_peered":"2024-09-11T05:16:20.848176+0000","last_clean":"2024-09-11T05:16:20.848176+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:20.848176+0000","last_undegraded":"2024-09-11T05:16:20.848176+0000","last_fullsized":"2024-09-11T05:16:20.848176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7813,"log_dups_size":0,"ondisk_log_size":7813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5716,"num_write_kb":22864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8156","reported_seq":8287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.636643+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:24.636643+0000","last_peered":"2024-09-11T05:16:24.636643+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:24.636643+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8156,"log_dups_size":0,"ondisk_log_size":8156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7880","reported_seq":8013,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.518197+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:24.518197+0000","last_peered":"2024-09-11T05:16:24.518197+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:24.518197+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7880,"log_dups_size":0,"ondisk_log_size":7880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32243712,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4746,"num_write_kb":18984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4377},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7982","reported_seq":8117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.624270+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:24.624270+0000","last_peered":"2024-09-11T05:16:24.624270+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:24.624270+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7982,"log_dups_size":0,"ondisk_log_size":7982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32677888,"num_objects":7978,"num_object_clones":0,"num_object_copies":15956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4740,"num_write_kb":18960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7923","reported_seq":8060,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.621908+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:24.621908+0000","last_peered":"2024-09-11T05:16:24.621908+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:24.621908+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7923,"log_dups_size":0,"ondisk_log_size":7923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32404073,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4930,"num_write_kb":19720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":522953321,"num_objects":127675,"num_object_clones":0,"num_object_copies":255350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":127675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90820,"num_write_kb":363280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15953920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1059434496,"data_stored":1059354479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127756,"ondisk_log_size":127756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592948,"kb_used_data":366884,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93778892,"statfs":{"total":96636764160,"available":96029585408,"internally_reserved":0,"allocated":375689216,"data_stored":375434362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405416,"kb_used_data":225176,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93966424,"statfs":{"total":96636764160,"available":96221618176,"internally_reserved":0,"allocated":230580224,"data_stored":230251713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184544939},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215151,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707544,"kb_used_data":441288,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93664296,"statfs":{"total":96636764160,"available":95912239104,"internally_reserved":0,"allocated":451878912,"data_stored":451619981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452038656,"data_stored":452035177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231145472,"data_stored":231072413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376250368,"data_stored":376246889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:27.343 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:28.071 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:28.073 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:28.113 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-11T05:16:27.426279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":526594684,"num_objects":128566,"num_object_clones":0,"num_object_copies":257132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":128566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91711,"num_write_kb":366838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":15953939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128648,"ondisk_log_size":128648,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1744508,"kb_used_data":1045580,"kb_used_omap":12,"kb_used_meta":698867,"kb_avail":281371012,"statfs":{"total":289910292480,"available":288123916288,"internally_reserved":0,"allocated":1070673920,"data_stored":1069831624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":715639809},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,4,5,9,16,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23728128,"num_objects":5793,"num_object_clones":0,"num_object_copies":11586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-129,"num_objects_unfound":0,"num_objects_dirty":5793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":528384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010272"},"pg_stats":[{"pgid":"2.f","version":"45'8128","reported_seq":8206,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.638467+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:24.638467+0000","last_peered":"2024-09-11T05:16:24.638467+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:24.638467+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8128,"log_dups_size":0,"ondisk_log_size":8128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33292288,"num_objects":8128,"num_object_clones":0,"num_object_copies":16256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8128,"num_write_kb":32512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8116","reported_seq":8218,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:25.921409+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:25.921409+0000","last_peered":"2024-09-11T05:16:25.921409+0000","last_clean":"2024-09-11T05:16:25.921409+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:25.921409+0000","last_undegraded":"2024-09-11T05:16:25.921409+0000","last_fullsized":"2024-09-11T05:16:25.921409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8116,"log_dups_size":0,"ondisk_log_size":8116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6514,"num_write_kb":26056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7986","reported_seq":8111,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:25.799289+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:25.799289+0000","last_peered":"2024-09-11T05:16:25.799289+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:25.799289+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7986,"log_dups_size":0,"ondisk_log_size":7986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694272,"num_objects":7982,"num_object_clones":0,"num_object_copies":15964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6389,"num_write_kb":25556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4488}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8119","reported_seq":8190,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:25.879630+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:25.879630+0000","last_peered":"2024-09-11T05:16:25.879630+0000","last_clean":"2024-09-11T05:16:25.879630+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:25.879630+0000","last_undegraded":"2024-09-11T05:16:25.879630+0000","last_fullsized":"2024-09-11T05:16:25.879630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8119,"log_dups_size":0,"ondisk_log_size":8119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8119,"num_write_kb":32476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8155","reported_seq":8284,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:24.593049+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:24.593049+0000","last_peered":"2024-09-11T05:16:24.593049+0000","last_clean":"2024-09-11T05:16:24.593049+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:24.593049+0000","last_undegraded":"2024-09-11T05:16:24.593049+0000","last_fullsized":"2024-09-11T05:16:24.593049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33374208,"num_objects":8148,"num_object_clones":0,"num_object_copies":16296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8087","reported_seq":9068,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.647320+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:24.647320+0000","last_peered":"2024-09-11T05:16:24.647320+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:24.647320+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8087,"log_dups_size":0,"ondisk_log_size":8087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4631,"num_write_kb":18524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4898},{"shards":"0,2","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8066","reported_seq":8198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.604488+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:24.604488+0000","last_peered":"2024-09-11T05:16:24.604488+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:24.604488+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33013760,"num_objects":8060,"num_object_clones":0,"num_object_copies":16120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5231,"num_write_kb":20924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8004","reported_seq":8128,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:24.603807+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:24.603807+0000","last_peered":"2024-09-11T05:16:24.603807+0000","last_clean":"2024-09-11T05:16:24.603807+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:24.603807+0000","last_undegraded":"2024-09-11T05:16:24.603807+0000","last_fullsized":"2024-09-11T05:16:24.603807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8004,"log_dups_size":0,"ondisk_log_size":8004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32776192,"num_objects":8002,"num_object_clones":0,"num_object_copies":16004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8123","reported_seq":8254,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:25.929820+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:25.929820+0000","last_peered":"2024-09-11T05:16:25.929820+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:25.929820+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8123,"log_dups_size":0,"ondisk_log_size":8123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243136,"num_objects":8116,"num_object_clones":0,"num_object_copies":16232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6191,"num_write_kb":24764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4591}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7874","reported_seq":8037,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.614517+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:24.614517+0000","last_peered":"2024-09-11T05:16:24.614517+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:24.614517+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7874,"log_dups_size":0,"ondisk_log_size":7874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4424},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8084","reported_seq":8201,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:25.889200+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:25.889200+0000","last_peered":"2024-09-11T05:16:25.889200+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:25.889200+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8084,"log_dups_size":0,"ondisk_log_size":8084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":24452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7962","reported_seq":15047,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:25.880986+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:25.880986+0000","last_peered":"2024-09-11T05:16:25.880986+0000","last_clean":"2024-09-11T05:16:25.880986+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:25.880986+0000","last_undegraded":"2024-09-11T05:16:25.880986+0000","last_fullsized":"2024-09-11T05:16:25.880986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32587776,"num_objects":7956,"num_object_clones":0,"num_object_copies":15912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5865,"num_write_kb":23460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8156","reported_seq":8287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.636643+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:24.636643+0000","last_peered":"2024-09-11T05:16:24.636643+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:24.636643+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8156,"log_dups_size":0,"ondisk_log_size":8156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7880","reported_seq":8013,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.518197+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:24.518197+0000","last_peered":"2024-09-11T05:16:24.518197+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:24.518197+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7880,"log_dups_size":0,"ondisk_log_size":7880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32243712,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4746,"num_write_kb":18984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4377},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7982","reported_seq":8117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.624270+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:24.624270+0000","last_peered":"2024-09-11T05:16:24.624270+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:24.624270+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7982,"log_dups_size":0,"ondisk_log_size":7982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32677888,"num_objects":7978,"num_object_clones":0,"num_object_copies":15956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4740,"num_write_kb":18960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7923","reported_seq":8060,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.621908+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:24.621908+0000","last_peered":"2024-09-11T05:16:24.621908+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:24.621908+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7923,"log_dups_size":0,"ondisk_log_size":7923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32404073,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4930,"num_write_kb":19720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":526594665,"num_objects":128564,"num_object_clones":0,"num_object_copies":257128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":128564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91709,"num_write_kb":366836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15953920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1072209920,"data_stored":1072129903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128645,"ondisk_log_size":128645,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601888,"kb_used_data":371664,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93769952,"statfs":{"total":96636764160,"available":96020430848,"internally_reserved":0,"allocated":380583936,"data_stored":380329082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435076,"kb_used_data":232628,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93936764,"statfs":{"total":96636764160,"available":96191246336,"internally_reserved":0,"allocated":238211072,"data_stored":237882561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":207285931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215151,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707544,"kb_used_data":441288,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93664296,"statfs":{"total":96636764160,"available":95912239104,"internally_reserved":0,"allocated":451878912,"data_stored":451619981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452038656,"data_stored":452035177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238956544,"data_stored":238883485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381214720,"data_stored":381211241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:28.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:28.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8128", 'reported_seq': 8206, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.638467+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:24.638467+0000', 'last_peered': '2024-09-11T05:16:24.638467+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:24.638467+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 8128, 'log_dups_size': 0, 'ondisk_log_size': 8128, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33292288, 'num_objects': 8128, 'num_object_clones': 0, 'num_object_copies': 16256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8128, 'num_write_kb': 32512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4453}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:28.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'7986", 'reported_seq': 8111, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:25.799289+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:25.799289+0000', 'last_peered': '2024-09-11T05:16:25.799289+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:25.799289+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 7986, 'log_dups_size': 0, 'ondisk_log_size': 7986, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32694272, 'num_objects': 7982, 'num_object_clones': 0, 'num_object_copies': 15964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6389, 'num_write_kb': 25556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 4488}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:28.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:28.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8087", 'reported_seq': 9068, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.647320+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:24.647320+0000', 'last_peered': '2024-09-11T05:16:24.647320+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:24.647320+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 8087, 'log_dups_size': 0, 'ondisk_log_size': 8087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33103872, 'num_objects': 8082, 'num_object_clones': 0, 'num_object_copies': 16164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3182, 'num_objects_unfound': 0, 'num_objects_dirty': 8082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4631, 'num_write_kb': 18524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 416, 'num_bytes_recovered': 1703936, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4898}, {'shards': '0,2', 'objects': 3182}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:28.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8066", 'reported_seq': 8198, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.604488+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:24.604488+0000', 'last_peered': '2024-09-11T05:16:24.604488+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:24.604488+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 8066, 'log_dups_size': 0, 'ondisk_log_size': 8066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33013760, 'num_objects': 8060, 'num_object_clones': 0, 'num_object_copies': 16120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8060, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5231, 'num_write_kb': 20924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4484}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:28.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8123", 'reported_seq': 8254, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:25.929820+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:25.929820+0000', 'last_peered': '2024-09-11T05:16:25.929820+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:25.929820+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 8123, 'log_dups_size': 0, 'ondisk_log_size': 8123, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33243136, 'num_objects': 8116, 'num_object_clones': 0, 'num_object_copies': 16232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8116, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6191, 'num_write_kb': 24764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4591}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'7874", 'reported_seq': 8037, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.614517+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:24.614517+0000', 'last_peered': '2024-09-11T05:16:24.614517+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:24.614517+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 7874, 'log_dups_size': 0, 'ondisk_log_size': 7874, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32219136, 'num_objects': 7866, 'num_object_clones': 0, 'num_object_copies': 15732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4627, 'num_write_kb': 18508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4424}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8084", 'reported_seq': 8201, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:25.889200+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:25.889200+0000', 'last_peered': '2024-09-11T05:16:25.889200+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:25.889200+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 8084, 'log_dups_size': 0, 'ondisk_log_size': 8084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33091584, 'num_objects': 8079, 'num_object_clones': 0, 'num_object_copies': 16158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8079, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6113, 'num_write_kb': 24452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 4557}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8156", 'reported_seq': 8287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.636643+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:24.636643+0000', 'last_peered': '2024-09-11T05:16:24.636643+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:24.636643+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 8156, 'log_dups_size': 0, 'ondisk_log_size': 8156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33398784, 'num_objects': 8154, 'num_object_clones': 0, 'num_object_copies': 16308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5189, 'num_write_kb': 20756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4531}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:28.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7880", 'reported_seq': 8013, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.518197+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:24.518197+0000', 'last_peered': '2024-09-11T05:16:24.518197+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:24.518197+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 7880, 'log_dups_size': 0, 'ondisk_log_size': 7880, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32243712, 'num_objects': 7872, 'num_object_clones': 0, 'num_object_copies': 15744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7872, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4746, 'num_write_kb': 18984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4377}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:28.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'7982", 'reported_seq': 8117, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.624270+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:24.624270+0000', 'last_peered': '2024-09-11T05:16:24.624270+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:24.624270+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 7982, 'log_dups_size': 0, 'ondisk_log_size': 7982, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32677888, 'num_objects': 7978, 'num_object_clones': 0, 'num_object_copies': 15956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4740, 'num_write_kb': 18960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4434}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:28.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'7923", 'reported_seq': 8060, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:24.621908+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:24.621908+0000', 'last_peered': '2024-09-11T05:16:24.621908+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:24.621908+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 7923, 'log_dups_size': 0, 'ondisk_log_size': 7923, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32404073, 'num_objects': 7912, 'num_object_clones': 0, 'num_object_copies': 15824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4930, 'num_write_kb': 19720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4397}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:28.118 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:28.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 175 20 130012 129992 2.90042 1.82031 0.0116595 0.0269269 2024-09-11T05:16:29.093 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:29.093 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/257132 objects degraded (13.779%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":266,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128566,"data_bytes":526594684,"bytes_used":1786376192,"bytes_avail":288123916288,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":257132,"degraded_ratio":0.13779304015058413,"misplaced_objects":3182,"misplaced_total":257132,"misplaced_ratio":0.012374966943048706,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":43994,"recovering_keys_per_sec":0,"num_objects_recovered":129,"num_bytes_recovered":528384,"num_keys_recovered":0,"write_bytes_sec":1975652,"read_op_per_sec":0,"write_op_per_sec":482},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:29.140 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 176 20 130479 130459 2.89426 1.82422 0.0149516 0.0269839 2024-09-11T05:16:29.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:29.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:30.141 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 177 20 130941 130921 2.88811 1.80469 0.0668778 0.0270426 2024-09-11T05:16:30.167 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:30.170 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:30.250 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-11T05:16:29.426771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":526594684,"num_objects":128566,"num_object_clones":0,"num_object_copies":257132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":128566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91711,"num_write_kb":366838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":15953939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128648,"ondisk_log_size":128648,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1744508,"kb_used_data":1045580,"kb_used_omap":12,"kb_used_meta":698867,"kb_avail":281371012,"statfs":{"total":289910292480,"available":288123916288,"internally_reserved":0,"allocated":1070673920,"data_stored":1069831624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":715639809},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,4,5,9,16,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-129,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4890,"num_write_kb":19560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":528384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010245"},"pg_stats":[{"pgid":"2.f","version":"45'8128","reported_seq":8206,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.638467+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:24.638467+0000","last_peered":"2024-09-11T05:16:24.638467+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:24.638467+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8128,"log_dups_size":0,"ondisk_log_size":8128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33292288,"num_objects":8128,"num_object_clones":0,"num_object_copies":16256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8128,"num_write_kb":32512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8116","reported_seq":8218,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:25.921409+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:25.921409+0000","last_peered":"2024-09-11T05:16:25.921409+0000","last_clean":"2024-09-11T05:16:25.921409+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:25.921409+0000","last_undegraded":"2024-09-11T05:16:25.921409+0000","last_fullsized":"2024-09-11T05:16:25.921409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8116,"log_dups_size":0,"ondisk_log_size":8116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6514,"num_write_kb":26056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7986","reported_seq":8111,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:25.799289+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:25.799289+0000","last_peered":"2024-09-11T05:16:25.799289+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:25.799289+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":7986,"log_dups_size":0,"ondisk_log_size":7986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694272,"num_objects":7982,"num_object_clones":0,"num_object_copies":15964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6389,"num_write_kb":25556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4488}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8119","reported_seq":8190,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:25.879630+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:25.879630+0000","last_peered":"2024-09-11T05:16:25.879630+0000","last_clean":"2024-09-11T05:16:25.879630+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:25.879630+0000","last_undegraded":"2024-09-11T05:16:25.879630+0000","last_fullsized":"2024-09-11T05:16:25.879630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8119,"log_dups_size":0,"ondisk_log_size":8119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8119,"num_write_kb":32476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8155","reported_seq":8284,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:24.593049+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:24.593049+0000","last_peered":"2024-09-11T05:16:24.593049+0000","last_clean":"2024-09-11T05:16:24.593049+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:24.593049+0000","last_undegraded":"2024-09-11T05:16:24.593049+0000","last_fullsized":"2024-09-11T05:16:24.593049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33374208,"num_objects":8148,"num_object_clones":0,"num_object_copies":16296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8087","reported_seq":9068,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.647320+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:24.647320+0000","last_peered":"2024-09-11T05:16:24.647320+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:24.647320+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8087,"log_dups_size":0,"ondisk_log_size":8087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33103872,"num_objects":8082,"num_object_clones":0,"num_object_copies":16164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":8082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4631,"num_write_kb":18524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4898},{"shards":"0,2","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8066","reported_seq":8198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.604488+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:24.604488+0000","last_peered":"2024-09-11T05:16:24.604488+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:24.604488+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33013760,"num_objects":8060,"num_object_clones":0,"num_object_copies":16120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5231,"num_write_kb":20924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8004","reported_seq":8128,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:24.603807+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:24.603807+0000","last_peered":"2024-09-11T05:16:24.603807+0000","last_clean":"2024-09-11T05:16:24.603807+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:24.603807+0000","last_undegraded":"2024-09-11T05:16:24.603807+0000","last_fullsized":"2024-09-11T05:16:24.603807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8004,"log_dups_size":0,"ondisk_log_size":8004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32776192,"num_objects":8002,"num_object_clones":0,"num_object_copies":16004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8123","reported_seq":8254,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:25.929820+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:25.929820+0000","last_peered":"2024-09-11T05:16:25.929820+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:25.929820+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8123,"log_dups_size":0,"ondisk_log_size":8123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243136,"num_objects":8116,"num_object_clones":0,"num_object_copies":16232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6191,"num_write_kb":24764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4591}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7874","reported_seq":8037,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.614517+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:24.614517+0000","last_peered":"2024-09-11T05:16:24.614517+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:24.614517+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":7874,"log_dups_size":0,"ondisk_log_size":7874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4424},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8084","reported_seq":8201,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:25.889200+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:25.889200+0000","last_peered":"2024-09-11T05:16:25.889200+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:25.889200+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8084,"log_dups_size":0,"ondisk_log_size":8084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":24452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7962","reported_seq":15047,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:25.880986+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:25.880986+0000","last_peered":"2024-09-11T05:16:25.880986+0000","last_clean":"2024-09-11T05:16:25.880986+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:25.880986+0000","last_undegraded":"2024-09-11T05:16:25.880986+0000","last_fullsized":"2024-09-11T05:16:25.880986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32587776,"num_objects":7956,"num_object_clones":0,"num_object_copies":15912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5865,"num_write_kb":23460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8156","reported_seq":8287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.636643+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:24.636643+0000","last_peered":"2024-09-11T05:16:24.636643+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:24.636643+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8156,"log_dups_size":0,"ondisk_log_size":8156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5189,"num_write_kb":20756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7880","reported_seq":8013,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.518197+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:24.518197+0000","last_peered":"2024-09-11T05:16:24.518197+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:24.518197+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":7880,"log_dups_size":0,"ondisk_log_size":7880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32243712,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4746,"num_write_kb":18984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4377},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7982","reported_seq":8117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.624270+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:24.624270+0000","last_peered":"2024-09-11T05:16:24.624270+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:24.624270+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":7982,"log_dups_size":0,"ondisk_log_size":7982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32677888,"num_objects":7978,"num_object_clones":0,"num_object_copies":15956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4740,"num_write_kb":18960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7923","reported_seq":8060,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:24.621908+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:24.621908+0000","last_peered":"2024-09-11T05:16:24.621908+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:24.621908+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":7923,"log_dups_size":0,"ondisk_log_size":7923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32404073,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4930,"num_write_kb":19720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":526594665,"num_objects":128564,"num_object_clones":0,"num_object_copies":257128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":128564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91709,"num_write_kb":366836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15953920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1072209920,"data_stored":1072129903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128645,"ondisk_log_size":128645,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601888,"kb_used_data":371664,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93769952,"statfs":{"total":96636764160,"available":96020430848,"internally_reserved":0,"allocated":380583936,"data_stored":380329082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435076,"kb_used_data":232628,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93936764,"statfs":{"total":96636764160,"available":96191246336,"internally_reserved":0,"allocated":238211072,"data_stored":237882561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":207285931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215151,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707544,"kb_used_data":441288,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93664296,"statfs":{"total":96636764160,"available":95912239104,"internally_reserved":0,"allocated":451878912,"data_stored":451619981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452038656,"data_stored":452035177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238956544,"data_stored":238883485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381214720,"data_stored":381211241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:31.138 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 178 20 131440 131420 2.88283 1.94922 0.0574064 0.0270925 2024-09-11T05:16:32.138 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 179 20 131918 131898 2.87716 1.86719 0.0184519 0.0271436 2024-09-11T05:16:33.139 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:16:33.140752+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0271949 2024-09-11T05:16:33.139 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:16:33.139 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 180 20 132419 132399 2.87204 1.95703 0.0300167 0.0271949 2024-09-11T05:16:33.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:34.112 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:34.117 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:34.139 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 181 20 132878 132858 2.86608 1.79297 0.0148954 0.0272506 2024-09-11T05:16:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-11T05:16:33.431201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":536470140,"num_objects":130977,"num_object_clones":0,"num_object_copies":261954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3116,"num_objects_unfound":0,"num_objects_dirty":130977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94122,"num_write_kb":376482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3964,"num_bytes_recovered":16224275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131059,"ondisk_log_size":131059,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786168,"kb_used_data":1065032,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281329352,"statfs":{"total":289910292480,"available":288081256448,"internally_reserved":0,"allocated":1090592768,"data_stored":1089750472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":738380801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,7,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19718144,"num_objects":4814,"num_object_clones":0,"num_object_copies":9628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-133,"num_objects_unfound":0,"num_objects_dirty":4814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4814,"num_write_kb":19256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":544768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013566"},"pg_stats":[{"pgid":"2.f","version":"45'8274","reported_seq":8352,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.662189+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:29.662189+0000","last_peered":"2024-09-11T05:16:29.662189+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:29.662189+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8274,"log_dups_size":0,"ondisk_log_size":8274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8274,"num_write_kb":33096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4599},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8272","reported_seq":8374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.890253+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:30.890253+0000","last_peered":"2024-09-11T05:16:30.890253+0000","last_clean":"2024-09-11T05:16:30.890253+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:30.890253+0000","last_undegraded":"2024-09-11T05:16:30.890253+0000","last_fullsized":"2024-09-11T05:16:30.890253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33857536,"num_objects":8266,"num_object_clones":0,"num_object_copies":16532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6670,"num_write_kb":26680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8140","reported_seq":8265,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.835573+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:30.835573+0000","last_peered":"2024-09-11T05:16:30.835573+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:30.835573+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8140,"log_dups_size":0,"ondisk_log_size":8140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6543,"num_write_kb":26172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4642}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8263","reported_seq":8334,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.941248+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:30.941248+0000","last_peered":"2024-09-11T05:16:30.941248+0000","last_clean":"2024-09-11T05:16:30.941248+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:30.941248+0000","last_undegraded":"2024-09-11T05:16:30.941248+0000","last_fullsized":"2024-09-11T05:16:30.941248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8263,"log_dups_size":0,"ondisk_log_size":8263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33845248,"num_objects":8263,"num_object_clones":0,"num_object_copies":16526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8263,"num_write_kb":33052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8319","reported_seq":8448,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:29.701728+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:29.701728+0000","last_peered":"2024-09-11T05:16:29.701728+0000","last_clean":"2024-09-11T05:16:29.701728+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:29.701728+0000","last_undegraded":"2024-09-11T05:16:29.701728+0000","last_fullsized":"2024-09-11T05:16:29.701728+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8319,"log_dups_size":0,"ondisk_log_size":8319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5153,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8215","reported_seq":9329,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.672559+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:29.672559+0000","last_peered":"2024-09-11T05:16:29.672559+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:29.672559+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8215,"log_dups_size":0,"ondisk_log_size":8215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33628160,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3116,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4759,"num_write_kb":19036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5092},{"shards":"0,2","objects":3116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8211","reported_seq":8343,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.710653+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:29.710653+0000","last_peered":"2024-09-11T05:16:29.710653+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:29.710653+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5376,"num_write_kb":21504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8131","reported_seq":8255,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:29.572345+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:29.572345+0000","last_peered":"2024-09-11T05:16:29.572345+0000","last_clean":"2024-09-11T05:16:29.572345+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:29.572345+0000","last_undegraded":"2024-09-11T05:16:29.572345+0000","last_fullsized":"2024-09-11T05:16:29.572345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8131,"log_dups_size":0,"ondisk_log_size":8131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5434,"num_write_kb":21736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8265","reported_seq":8396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.860570+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:30.860570+0000","last_peered":"2024-09-11T05:16:30.860570+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:30.860570+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8265,"log_dups_size":0,"ondisk_log_size":8265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33824768,"num_objects":8258,"num_object_clones":0,"num_object_copies":16516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":25332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4733}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8051","reported_seq":8216,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.587902+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:29.587902+0000","last_peered":"2024-09-11T05:16:29.587902+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:29.587902+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4804,"num_write_kb":19216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4601},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8231","reported_seq":8348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.910062+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:30.910062+0000","last_peered":"2024-09-11T05:16:30.910062+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:30.910062+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8231,"log_dups_size":0,"ondisk_log_size":8231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33693696,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8110","reported_seq":15195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.851854+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:30.851854+0000","last_peered":"2024-09-11T05:16:30.851854+0000","last_clean":"2024-09-11T05:16:30.851854+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:30.851854+0000","last_undegraded":"2024-09-11T05:16:30.851854+0000","last_fullsized":"2024-09-11T05:16:30.851854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8110,"log_dups_size":0,"ondisk_log_size":8110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33193984,"num_objects":8104,"num_object_clones":0,"num_object_copies":16208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":24052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8325","reported_seq":8453,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.730264+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:29.730264+0000","last_peered":"2024-09-11T05:16:29.730264+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:29.730264+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8325,"log_dups_size":0,"ondisk_log_size":8325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34091008,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":21432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4700},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8027","reported_seq":8160,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.689346+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:29.689346+0000","last_peered":"2024-09-11T05:16:29.689346+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:29.689346+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8027,"log_dups_size":0,"ondisk_log_size":8027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32845824,"num_objects":8019,"num_object_clones":0,"num_object_copies":16038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4893,"num_write_kb":19572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4524},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8140","reported_seq":8275,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.695466+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:29.695466+0000","last_peered":"2024-09-11T05:16:29.695466+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:29.695466+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8140,"log_dups_size":0,"ondisk_log_size":8140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4898,"num_write_kb":19592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8082","reported_seq":8225,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.629174+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:29.629174+0000","last_peered":"2024-09-11T05:16:29.629174+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:29.629174+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8082,"log_dups_size":0,"ondisk_log_size":8082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33055337,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5089,"num_write_kb":20356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4556},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":536470121,"num_objects":130975,"num_object_clones":0,"num_object_copies":261950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3116,"num_objects_unfound":0,"num_objects_dirty":130975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94120,"num_write_kb":376480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3963,"num_bytes_recovered":16224256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092231168,"data_stored":1092151151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131056,"ondisk_log_size":131056,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610776,"kb_used_data":376392,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93761064,"statfs":{"total":96636764160,"available":96011329536,"internally_reserved":0,"allocated":385425408,"data_stored":385170554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434136,"kb_used_data":240008,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93937704,"statfs":{"total":96636764160,"available":96192208896,"internally_reserved":0,"allocated":245768192,"data_stored":245439681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741256,"kb_used_data":448632,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93630584,"statfs":{"total":96636764160,"available":95877718016,"internally_reserved":0,"allocated":459399168,"data_stored":459140237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299626155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459644928,"data_stored":459641449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246591488,"data_stored":246518429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385994752,"data_stored":385991273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:34.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:34.951 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:35.022 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-11T05:16:33.431201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":536470140,"num_objects":130977,"num_object_clones":0,"num_object_copies":261954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3116,"num_objects_unfound":0,"num_objects_dirty":130977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94122,"num_write_kb":376482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3964,"num_bytes_recovered":16224275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131059,"ondisk_log_size":131059,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786168,"kb_used_data":1065032,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281329352,"statfs":{"total":289910292480,"available":288081256448,"internally_reserved":0,"allocated":1090592768,"data_stored":1089750472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":738380801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,7,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19718144,"num_objects":4814,"num_object_clones":0,"num_object_copies":9628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-133,"num_objects_unfound":0,"num_objects_dirty":4814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4814,"num_write_kb":19256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":544768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013566"},"pg_stats":[{"pgid":"2.f","version":"45'8274","reported_seq":8352,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.662189+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:29.662189+0000","last_peered":"2024-09-11T05:16:29.662189+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:29.662189+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8274,"log_dups_size":0,"ondisk_log_size":8274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8274,"num_write_kb":33096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4599},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8272","reported_seq":8374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.890253+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:30.890253+0000","last_peered":"2024-09-11T05:16:30.890253+0000","last_clean":"2024-09-11T05:16:30.890253+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:30.890253+0000","last_undegraded":"2024-09-11T05:16:30.890253+0000","last_fullsized":"2024-09-11T05:16:30.890253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33857536,"num_objects":8266,"num_object_clones":0,"num_object_copies":16532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6670,"num_write_kb":26680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8140","reported_seq":8265,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.835573+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:30.835573+0000","last_peered":"2024-09-11T05:16:30.835573+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:30.835573+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8140,"log_dups_size":0,"ondisk_log_size":8140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6543,"num_write_kb":26172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4642}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8263","reported_seq":8334,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.941248+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:30.941248+0000","last_peered":"2024-09-11T05:16:30.941248+0000","last_clean":"2024-09-11T05:16:30.941248+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:30.941248+0000","last_undegraded":"2024-09-11T05:16:30.941248+0000","last_fullsized":"2024-09-11T05:16:30.941248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8263,"log_dups_size":0,"ondisk_log_size":8263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33845248,"num_objects":8263,"num_object_clones":0,"num_object_copies":16526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8263,"num_write_kb":33052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8319","reported_seq":8448,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:29.701728+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:29.701728+0000","last_peered":"2024-09-11T05:16:29.701728+0000","last_clean":"2024-09-11T05:16:29.701728+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:29.701728+0000","last_undegraded":"2024-09-11T05:16:29.701728+0000","last_fullsized":"2024-09-11T05:16:29.701728+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8319,"log_dups_size":0,"ondisk_log_size":8319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5153,"num_write_kb":20612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8215","reported_seq":9329,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.672559+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:29.672559+0000","last_peered":"2024-09-11T05:16:29.672559+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:29.672559+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8215,"log_dups_size":0,"ondisk_log_size":8215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33628160,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3116,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4759,"num_write_kb":19036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5092},{"shards":"0,2","objects":3116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8211","reported_seq":8343,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.710653+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:29.710653+0000","last_peered":"2024-09-11T05:16:29.710653+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:29.710653+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5376,"num_write_kb":21504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8131","reported_seq":8255,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:29.572345+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:29.572345+0000","last_peered":"2024-09-11T05:16:29.572345+0000","last_clean":"2024-09-11T05:16:29.572345+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:29.572345+0000","last_undegraded":"2024-09-11T05:16:29.572345+0000","last_fullsized":"2024-09-11T05:16:29.572345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8131,"log_dups_size":0,"ondisk_log_size":8131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33296384,"num_objects":8129,"num_object_clones":0,"num_object_copies":16258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5434,"num_write_kb":21736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8265","reported_seq":8396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.860570+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:30.860570+0000","last_peered":"2024-09-11T05:16:30.860570+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:30.860570+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8265,"log_dups_size":0,"ondisk_log_size":8265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33824768,"num_objects":8258,"num_object_clones":0,"num_object_copies":16516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":25332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4733}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8051","reported_seq":8216,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.587902+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:29.587902+0000","last_peered":"2024-09-11T05:16:29.587902+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:29.587902+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4804,"num_write_kb":19216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4601},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8231","reported_seq":8348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.910062+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:30.910062+0000","last_peered":"2024-09-11T05:16:30.910062+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:30.910062+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8231,"log_dups_size":0,"ondisk_log_size":8231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33693696,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8110","reported_seq":15195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.851854+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:30.851854+0000","last_peered":"2024-09-11T05:16:30.851854+0000","last_clean":"2024-09-11T05:16:30.851854+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:30.851854+0000","last_undegraded":"2024-09-11T05:16:30.851854+0000","last_fullsized":"2024-09-11T05:16:30.851854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8110,"log_dups_size":0,"ondisk_log_size":8110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33193984,"num_objects":8104,"num_object_clones":0,"num_object_copies":16208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":24052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8325","reported_seq":8453,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.730264+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:29.730264+0000","last_peered":"2024-09-11T05:16:29.730264+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:29.730264+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8325,"log_dups_size":0,"ondisk_log_size":8325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34091008,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":21432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4700},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8027","reported_seq":8160,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.689346+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:29.689346+0000","last_peered":"2024-09-11T05:16:29.689346+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:29.689346+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8027,"log_dups_size":0,"ondisk_log_size":8027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32845824,"num_objects":8019,"num_object_clones":0,"num_object_copies":16038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4893,"num_write_kb":19572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4524},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8140","reported_seq":8275,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.695466+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:29.695466+0000","last_peered":"2024-09-11T05:16:29.695466+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:29.695466+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8140,"log_dups_size":0,"ondisk_log_size":8140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4898,"num_write_kb":19592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8082","reported_seq":8225,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:29.629174+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:29.629174+0000","last_peered":"2024-09-11T05:16:29.629174+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:29.629174+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8082,"log_dups_size":0,"ondisk_log_size":8082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33055337,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5089,"num_write_kb":20356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4556},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":536470121,"num_objects":130975,"num_object_clones":0,"num_object_copies":261950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3116,"num_objects_unfound":0,"num_objects_dirty":130975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94120,"num_write_kb":376480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3963,"num_bytes_recovered":16224256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092231168,"data_stored":1092151151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131056,"ondisk_log_size":131056,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610776,"kb_used_data":376392,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93761064,"statfs":{"total":96636764160,"available":96011329536,"internally_reserved":0,"allocated":385425408,"data_stored":385170554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434136,"kb_used_data":240008,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93937704,"statfs":{"total":96636764160,"available":96192208896,"internally_reserved":0,"allocated":245768192,"data_stored":245439681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741256,"kb_used_data":448632,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93630584,"statfs":{"total":96636764160,"available":95877718016,"internally_reserved":0,"allocated":459399168,"data_stored":459140237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299626155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459644928,"data_stored":459641449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246591488,"data_stored":246518429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385994752,"data_stored":385991273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:35.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:35.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8274", 'reported_seq': 8352, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.662189+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:29.662189+0000', 'last_peered': '2024-09-11T05:16:29.662189+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:29.662189+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 8274, 'log_dups_size': 0, 'ondisk_log_size': 8274, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33890304, 'num_objects': 8274, 'num_object_clones': 0, 'num_object_copies': 16548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8274, 'num_write_kb': 33096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4599}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:35.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8140", 'reported_seq': 8265, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:30.835573+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:30.835573+0000', 'last_peered': '2024-09-11T05:16:30.835573+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:30.835573+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 8140, 'log_dups_size': 0, 'ondisk_log_size': 8140, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33325056, 'num_objects': 8136, 'num_object_clones': 0, 'num_object_copies': 16272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6543, 'num_write_kb': 26172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 4642}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:35.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:35.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8215", 'reported_seq': 9329, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.672559+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:29.672559+0000', 'last_peered': '2024-09-11T05:16:29.672559+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:29.672559+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 8215, 'log_dups_size': 0, 'ondisk_log_size': 8215, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33628160, 'num_objects': 8210, 'num_object_clones': 0, 'num_object_copies': 16420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3116, 'num_objects_unfound': 0, 'num_objects_dirty': 8210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4759, 'num_write_kb': 19036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 482, 'num_bytes_recovered': 1974272, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5092}, {'shards': '0,2', 'objects': 3116}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:35.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8211", 'reported_seq': 8343, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.710653+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:29.710653+0000', 'last_peered': '2024-09-11T05:16:29.710653+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:29.710653+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 8211, 'log_dups_size': 0, 'ondisk_log_size': 8211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33607680, 'num_objects': 8205, 'num_object_clones': 0, 'num_object_copies': 16410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5376, 'num_write_kb': 21504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4629}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:35.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8265", 'reported_seq': 8396, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:30.860570+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:30.860570+0000', 'last_peered': '2024-09-11T05:16:30.860570+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:30.860570+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 8265, 'log_dups_size': 0, 'ondisk_log_size': 8265, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33824768, 'num_objects': 8258, 'num_object_clones': 0, 'num_object_copies': 16516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6333, 'num_write_kb': 25332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 4733}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:35.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:35.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8051", 'reported_seq': 8216, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.587902+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:29.587902+0000', 'last_peered': '2024-09-11T05:16:29.587902+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:29.587902+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 8051, 'log_dups_size': 0, 'ondisk_log_size': 8051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32944128, 'num_objects': 8043, 'num_object_clones': 0, 'num_object_copies': 16086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8043, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4804, 'num_write_kb': 19216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4601}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:35.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8231", 'reported_seq': 8348, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:30.910062+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:30.910062+0000', 'last_peered': '2024-09-11T05:16:30.910062+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:30.910062+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 8231, 'log_dups_size': 0, 'ondisk_log_size': 8231, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33693696, 'num_objects': 8226, 'num_object_clones': 0, 'num_object_copies': 16452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8226, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6260, 'num_write_kb': 25040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 4704}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:35.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:35.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8325", 'reported_seq': 8453, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.730264+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:29.730264+0000', 'last_peered': '2024-09-11T05:16:29.730264+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:29.730264+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 8325, 'log_dups_size': 0, 'ondisk_log_size': 8325, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34091008, 'num_objects': 8323, 'num_object_clones': 0, 'num_object_copies': 16646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8323, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5358, 'num_write_kb': 21432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4700}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:35.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8027", 'reported_seq': 8160, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.689346+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:29.689346+0000', 'last_peered': '2024-09-11T05:16:29.689346+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:29.689346+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 8027, 'log_dups_size': 0, 'ondisk_log_size': 8027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32845824, 'num_objects': 8019, 'num_object_clones': 0, 'num_object_copies': 16038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4893, 'num_write_kb': 19572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4524}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:35.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8140", 'reported_seq': 8275, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.695466+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:29.695466+0000', 'last_peered': '2024-09-11T05:16:29.695466+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:29.695466+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 8140, 'log_dups_size': 0, 'ondisk_log_size': 8140, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33325056, 'num_objects': 8136, 'num_object_clones': 0, 'num_object_copies': 16272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4898, 'num_write_kb': 19592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4592}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:35.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8082", 'reported_seq': 8225, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:29.629174+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:29.629174+0000', 'last_peered': '2024-09-11T05:16:29.629174+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:29.629174+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 8082, 'log_dups_size': 0, 'ondisk_log_size': 8082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33055337, 'num_objects': 8071, 'num_object_clones': 0, 'num_object_copies': 16142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5089, 'num_write_kb': 20356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4556}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:35.029 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:35.139 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 182 20 133328 133308 2.85999 1.75781 0.0244636 0.0273078 2024-09-11T05:16:35.827 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:35.827 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/264974 objects degraded (13.372%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":273,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132487,"data_bytes":542655100,"bytes_used":1823444992,"bytes_avail":288086847488,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":264974,"degraded_ratio":0.13371500600058875,"misplaced_objects":3049,"misplaced_total":264974,"misplaced_ratio":0.011506789345369734,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":68189,"recovering_keys_per_sec":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"write_bytes_sec":2156165,"read_op_per_sec":0,"write_op_per_sec":526},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:35.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:35.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:36.139 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 183 20 133770 133750 2.8538 1.72656 0.0244007 0.0273652 2024-09-11T05:16:36.768 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:36.769 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:36.835 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-11T05:16:35.431695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":542655100,"num_objects":132487,"num_object_clones":0,"num_object_copies":264974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3049,"num_objects_unfound":0,"num_objects_dirty":132487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95632,"num_write_kb":382522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4031,"num_bytes_recovered":16498707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132569,"ondisk_log_size":132569,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1780708,"kb_used_data":1072052,"kb_used_omap":12,"kb_used_meta":708595,"kb_avail":281334812,"statfs":{"total":289910292480,"available":288086847488,"internally_reserved":0,"allocated":1097781248,"data_stored":1096938952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":725601281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,7,5,11,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25903104,"num_objects":6324,"num_object_clones":0,"num_object_copies":12648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-200,"num_objects_unfound":0,"num_objects_dirty":6324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6324,"num_write_kb":25296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013506"},"pg_stats":[{"pgid":"2.f","version":"45'8409","reported_seq":8487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.666289+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:34.666289+0000","last_peered":"2024-09-11T05:16:34.666289+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:34.666289+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8409,"log_dups_size":0,"ondisk_log_size":8409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8409,"num_write_kb":33636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4734},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8272","reported_seq":8374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.890253+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:30.890253+0000","last_peered":"2024-09-11T05:16:30.890253+0000","last_clean":"2024-09-11T05:16:30.890253+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:30.890253+0000","last_undegraded":"2024-09-11T05:16:30.890253+0000","last_fullsized":"2024-09-11T05:16:30.890253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33857536,"num_objects":8266,"num_object_clones":0,"num_object_copies":16532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6670,"num_write_kb":26680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8140","reported_seq":8265,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.835573+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:30.835573+0000","last_peered":"2024-09-11T05:16:30.835573+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:30.835573+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8140,"log_dups_size":0,"ondisk_log_size":8140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6543,"num_write_kb":26172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4642}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8263","reported_seq":8334,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.941248+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:30.941248+0000","last_peered":"2024-09-11T05:16:30.941248+0000","last_clean":"2024-09-11T05:16:30.941248+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:30.941248+0000","last_undegraded":"2024-09-11T05:16:30.941248+0000","last_fullsized":"2024-09-11T05:16:30.941248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8263,"log_dups_size":0,"ondisk_log_size":8263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33845248,"num_objects":8263,"num_object_clones":0,"num_object_copies":16526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8263,"num_write_kb":33052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8476","reported_seq":8605,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:34.766466+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:34.766466+0000","last_peered":"2024-09-11T05:16:34.766466+0000","last_clean":"2024-09-11T05:16:34.766466+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:34.766466+0000","last_undegraded":"2024-09-11T05:16:34.766466+0000","last_fullsized":"2024-09-11T05:16:34.766466+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8476,"log_dups_size":0,"ondisk_log_size":8476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5310,"num_write_kb":21240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8367","reported_seq":9613,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.776370+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:34.776370+0000","last_peered":"2024-09-11T05:16:34.776370+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:34.776370+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8367,"log_dups_size":0,"ondisk_log_size":8367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34250752,"num_objects":8362,"num_object_clones":0,"num_object_copies":16724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3049,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4911,"num_write_kb":19644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":2248704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":3049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8354","reported_seq":8486,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.683784+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:34.683784+0000","last_peered":"2024-09-11T05:16:34.683784+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:34.683784+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8354,"log_dups_size":0,"ondisk_log_size":8354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34193408,"num_objects":8348,"num_object_clones":0,"num_object_copies":16696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5519,"num_write_kb":22076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8298","reported_seq":8422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:34.782580+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:34.782580+0000","last_peered":"2024-09-11T05:16:34.782580+0000","last_clean":"2024-09-11T05:16:34.782580+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:34.782580+0000","last_undegraded":"2024-09-11T05:16:34.782580+0000","last_fullsized":"2024-09-11T05:16:34.782580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8298,"log_dups_size":0,"ondisk_log_size":8298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5601,"num_write_kb":22404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8265","reported_seq":8396,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.860570+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:30.860570+0000","last_peered":"2024-09-11T05:16:30.860570+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:30.860570+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8265,"log_dups_size":0,"ondisk_log_size":8265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33824768,"num_objects":8258,"num_object_clones":0,"num_object_copies":16516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":25332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4733}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8219","reported_seq":8382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.827863+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:34.827863+0000","last_peered":"2024-09-11T05:16:34.827863+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:34.827863+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8219,"log_dups_size":0,"ondisk_log_size":8219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33632256,"num_objects":8211,"num_object_clones":0,"num_object_copies":16422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4972,"num_write_kb":19888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4769},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8231","reported_seq":8348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:30.910062+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:30.910062+0000","last_peered":"2024-09-11T05:16:30.910062+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:30.910062+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8231,"log_dups_size":0,"ondisk_log_size":8231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33693696,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8110","reported_seq":15195,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:30.851854+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:30.851854+0000","last_peered":"2024-09-11T05:16:30.851854+0000","last_clean":"2024-09-11T05:16:30.851854+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:30.851854+0000","last_undegraded":"2024-09-11T05:16:30.851854+0000","last_fullsized":"2024-09-11T05:16:30.851854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8110,"log_dups_size":0,"ondisk_log_size":8110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33193984,"num_objects":8104,"num_object_clones":0,"num_object_copies":16208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":24052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8483","reported_seq":8614,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.759357+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:34.759357+0000","last_peered":"2024-09-11T05:16:34.759357+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:34.759357+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8483,"log_dups_size":0,"ondisk_log_size":8483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34738176,"num_objects":8481,"num_object_clones":0,"num_object_copies":16962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5516,"num_write_kb":22064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4858},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8183","reported_seq":8316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.759121+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:34.759121+0000","last_peered":"2024-09-11T05:16:34.759121+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:34.759121+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8183,"log_dups_size":0,"ondisk_log_size":8183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33484800,"num_objects":8175,"num_object_clones":0,"num_object_copies":16350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5049,"num_write_kb":20196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4680},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8276","reported_seq":8411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.752924+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:34.752924+0000","last_peered":"2024-09-11T05:16:34.752924+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:34.752924+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8276,"log_dups_size":0,"ondisk_log_size":8276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33882112,"num_objects":8272,"num_object_clones":0,"num_object_copies":16544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5034,"num_write_kb":20136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8220","reported_seq":8363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.810465+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:34.810465+0000","last_peered":"2024-09-11T05:16:34.810465+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:34.810465+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33620585,"num_objects":8209,"num_object_clones":0,"num_object_copies":16418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4694},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":542655081,"num_objects":132485,"num_object_clones":0,"num_object_copies":264970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3049,"num_objects_unfound":0,"num_objects_dirty":132485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95630,"num_write_kb":382520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4030,"num_bytes_recovered":16498688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1099587584,"data_stored":1099507567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132566,"ondisk_log_size":132566,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610776,"kb_used_data":376392,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93761064,"statfs":{"total":96636764160,"available":96011329536,"internally_reserved":0,"allocated":385425408,"data_stored":385170554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434136,"kb_used_data":240008,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93937704,"statfs":{"total":96636764160,"available":96192208896,"internally_reserved":0,"allocated":245768192,"data_stored":245439681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735796,"kb_used_data":455652,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93636044,"statfs":{"total":96636764160,"available":95883309056,"internally_reserved":0,"allocated":466587648,"data_stored":466328717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,2,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467001344,"data_stored":466997865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246591488,"data_stored":246518429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385994752,"data_stored":385991273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:37.139 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 184 20 134236 134216 2.84818 1.82031 0.0466761 0.0274232 2024-09-11T05:16:38.140 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 185 20 134716 134696 2.84291 1.875 0.00703312 0.027472 2024-09-11T05:16:39.140 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 186 20 135216 135196 2.83813 1.95312 0.0420132 0.0275196 2024-09-11T05:16:39.837 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:40.140 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 187 20 135687 135667 2.83279 1.83984 0.0300452 0.0275692 2024-09-11T05:16:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:40.628 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:40.730 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-11T05:16:39.432703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546181756,"num_objects":133348,"num_object_clones":0,"num_object_copies":266696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3049,"num_objects_unfound":0,"num_objects_dirty":133348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96493,"num_write_kb":385966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4031,"num_bytes_recovered":16498707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133430,"ondisk_log_size":133430,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1805180,"kb_used_data":1084044,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281310340,"statfs":{"total":289910292480,"available":288061788160,"internally_reserved":0,"allocated":1110061056,"data_stored":1109218760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":738380801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,5,5,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19587072,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-133,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4782,"num_write_kb":19128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":544768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006423"},"pg_stats":[{"pgid":"2.f","version":"45'8409","reported_seq":8487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.666289+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:34.666289+0000","last_peered":"2024-09-11T05:16:34.666289+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:34.666289+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8409,"log_dups_size":0,"ondisk_log_size":8409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8409,"num_write_kb":33636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4734},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8415","reported_seq":8517,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:35.890118+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:35.890118+0000","last_peered":"2024-09-11T05:16:35.890118+0000","last_clean":"2024-09-11T05:16:35.890118+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:35.890118+0000","last_undegraded":"2024-09-11T05:16:35.890118+0000","last_fullsized":"2024-09-11T05:16:35.890118+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6813,"num_write_kb":27252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8262","reported_seq":8387,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:35.867029+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:35.867029+0000","last_peered":"2024-09-11T05:16:35.867029+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:35.867029+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8262,"log_dups_size":0,"ondisk_log_size":8262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33824768,"num_objects":8258,"num_object_clones":0,"num_object_copies":16516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8390","reported_seq":8461,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:35.859088+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:35.859088+0000","last_peered":"2024-09-11T05:16:35.859088+0000","last_clean":"2024-09-11T05:16:35.859088+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:35.859088+0000","last_undegraded":"2024-09-11T05:16:35.859088+0000","last_fullsized":"2024-09-11T05:16:35.859088+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8390,"log_dups_size":0,"ondisk_log_size":8390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8390,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8476","reported_seq":8605,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:34.766466+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:34.766466+0000","last_peered":"2024-09-11T05:16:34.766466+0000","last_clean":"2024-09-11T05:16:34.766466+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:34.766466+0000","last_undegraded":"2024-09-11T05:16:34.766466+0000","last_fullsized":"2024-09-11T05:16:34.766466+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8476,"log_dups_size":0,"ondisk_log_size":8476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5310,"num_write_kb":21240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8367","reported_seq":9613,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.776370+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:34.776370+0000","last_peered":"2024-09-11T05:16:34.776370+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:34.776370+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8367,"log_dups_size":0,"ondisk_log_size":8367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34250752,"num_objects":8362,"num_object_clones":0,"num_object_copies":16724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3049,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4911,"num_write_kb":19644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":2248704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":3049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8354","reported_seq":8486,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.683784+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:34.683784+0000","last_peered":"2024-09-11T05:16:34.683784+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:34.683784+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8354,"log_dups_size":0,"ondisk_log_size":8354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34193408,"num_objects":8348,"num_object_clones":0,"num_object_copies":16696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5519,"num_write_kb":22076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8298","reported_seq":8422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:34.782580+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:34.782580+0000","last_peered":"2024-09-11T05:16:34.782580+0000","last_clean":"2024-09-11T05:16:34.782580+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:34.782580+0000","last_undegraded":"2024-09-11T05:16:34.782580+0000","last_fullsized":"2024-09-11T05:16:34.782580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8298,"log_dups_size":0,"ondisk_log_size":8298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5601,"num_write_kb":22404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8417","reported_seq":8548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:35.944620+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:35.944620+0000","last_peered":"2024-09-11T05:16:35.944620+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:35.944620+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8417,"log_dups_size":0,"ondisk_log_size":8417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":4885}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8219","reported_seq":8382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.827863+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:34.827863+0000","last_peered":"2024-09-11T05:16:34.827863+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:34.827863+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8219,"log_dups_size":0,"ondisk_log_size":8219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33632256,"num_objects":8211,"num_object_clones":0,"num_object_copies":16422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4972,"num_write_kb":19888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4769},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8405","reported_seq":8522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:35.887860+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:35.887860+0000","last_peered":"2024-09-11T05:16:35.887860+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:35.887860+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8405,"log_dups_size":0,"ondisk_log_size":8405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34406400,"num_objects":8400,"num_object_clones":0,"num_object_copies":16800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":4878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8253","reported_seq":15338,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:35.945221+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:35.945221+0000","last_peered":"2024-09-11T05:16:35.945221+0000","last_clean":"2024-09-11T05:16:35.945221+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:35.945221+0000","last_undegraded":"2024-09-11T05:16:35.945221+0000","last_fullsized":"2024-09-11T05:16:35.945221+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8253,"log_dups_size":0,"ondisk_log_size":8253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33779712,"num_objects":8247,"num_object_clones":0,"num_object_copies":16494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8483","reported_seq":8614,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.759357+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:34.759357+0000","last_peered":"2024-09-11T05:16:34.759357+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:34.759357+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8483,"log_dups_size":0,"ondisk_log_size":8483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34738176,"num_objects":8481,"num_object_clones":0,"num_object_copies":16962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5516,"num_write_kb":22064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4858},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8183","reported_seq":8316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.759121+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:34.759121+0000","last_peered":"2024-09-11T05:16:34.759121+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:34.759121+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8183,"log_dups_size":0,"ondisk_log_size":8183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33484800,"num_objects":8175,"num_object_clones":0,"num_object_copies":16350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5049,"num_write_kb":20196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4680},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8276","reported_seq":8411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.752924+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:34.752924+0000","last_peered":"2024-09-11T05:16:34.752924+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:34.752924+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8276,"log_dups_size":0,"ondisk_log_size":8276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33882112,"num_objects":8272,"num_object_clones":0,"num_object_copies":16544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5034,"num_write_kb":20136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8220","reported_seq":8363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:34.810465+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:34.810465+0000","last_peered":"2024-09-11T05:16:34.810465+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:34.810465+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8220,"log_dups_size":0,"ondisk_log_size":8220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33620585,"num_objects":8209,"num_object_clones":0,"num_object_copies":16418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4694},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546181737,"num_objects":133346,"num_object_clones":0,"num_object_copies":266692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":3049,"num_objects_unfound":0,"num_objects_dirty":133346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96491,"num_write_kb":385964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4030,"num_bytes_recovered":16498688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1111920640,"data_stored":1111840623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133427,"ondisk_log_size":133427,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623980,"kb_used_data":381276,"kb_used_omap":4,"kb_used_meta":242683,"kb_avail":93747860,"statfs":{"total":96636764160,"available":95997808640,"internally_reserved":0,"allocated":390426624,"data_stored":390171770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":248508075},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445404,"kb_used_data":247116,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93926436,"statfs":{"total":96636764160,"available":96180670464,"internally_reserved":0,"allocated":253046784,"data_stored":252718273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735796,"kb_used_data":455652,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93636044,"statfs":{"total":96636764160,"available":95883309056,"internally_reserved":0,"allocated":466587648,"data_stored":466328717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,2,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467001344,"data_stored":466997865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":254083072,"data_stored":254010013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390836224,"data_stored":390832745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:40.732 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:41.140 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 188 20 136150 136130 2.82735 1.80859 0.0373284 0.0276248 2024-09-11T05:16:41.588 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:41.590 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:41.647 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-11T05:16:41.433160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555987580,"num_objects":135742,"num_object_clones":0,"num_object_copies":271484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2985,"num_objects_unfound":0,"num_objects_dirty":135742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98887,"num_write_kb":395542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4095,"num_bytes_recovered":16760851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135824,"ondisk_log_size":135824,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1826364,"kb_used_data":1095500,"kb_used_omap":12,"kb_used_meta":730803,"kb_avail":281289156,"statfs":{"total":289910292480,"available":288040095744,"internally_reserved":0,"allocated":1121792000,"data_stored":1120949704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":748342273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,5,8,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-197,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7176,"num_write_kb":28704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006389"},"pg_stats":[{"pgid":"2.f","version":"45'8583","reported_seq":8661,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.843679+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:39.843679+0000","last_peered":"2024-09-11T05:16:39.843679+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:39.843679+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8583,"log_dups_size":0,"ondisk_log_size":8583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35155968,"num_objects":8583,"num_object_clones":0,"num_object_copies":17166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":34332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4908},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8542","reported_seq":8644,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:40.971577+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:40.971577+0000","last_peered":"2024-09-11T05:16:40.971577+0000","last_clean":"2024-09-11T05:16:40.971577+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:40.971577+0000","last_undegraded":"2024-09-11T05:16:40.971577+0000","last_fullsized":"2024-09-11T05:16:40.971577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34963456,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6940,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8399","reported_seq":8524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:40.875389+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:40.875389+0000","last_peered":"2024-09-11T05:16:40.875389+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:40.875389+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8399,"log_dups_size":0,"ondisk_log_size":8399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34385920,"num_objects":8395,"num_object_clones":0,"num_object_copies":16790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6802,"num_write_kb":27208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4901}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8540","reported_seq":8610,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:40.957496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:40.957496+0000","last_peered":"2024-09-11T05:16:40.957496+0000","last_clean":"2024-09-11T05:16:40.957496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:40.957496+0000","last_undegraded":"2024-09-11T05:16:40.957496+0000","last_fullsized":"2024-09-11T05:16:40.957496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34979840,"num_objects":8540,"num_object_clones":0,"num_object_copies":17080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8540,"num_write_kb":34160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8616","reported_seq":8745,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:39.752698+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:39.752698+0000","last_peered":"2024-09-11T05:16:39.752698+0000","last_clean":"2024-09-11T05:16:39.752698+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:39.752698+0000","last_undegraded":"2024-09-11T05:16:39.752698+0000","last_fullsized":"2024-09-11T05:16:39.752698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8616,"log_dups_size":0,"ondisk_log_size":8616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5450,"num_write_kb":21800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8532","reported_seq":9908,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.891322+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:39.891322+0000","last_peered":"2024-09-11T05:16:39.891322+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:39.891322+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8532,"log_dups_size":0,"ondisk_log_size":8532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34926592,"num_objects":8527,"num_object_clones":0,"num_object_copies":17054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2985,"num_objects_unfound":0,"num_objects_dirty":8527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5076,"num_write_kb":20304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":613,"num_bytes_recovered":2510848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5540},{"shards":"0,2","objects":2985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8506","reported_seq":8638,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.915322+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:39.915322+0000","last_peered":"2024-09-11T05:16:39.915322+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:39.915322+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8506,"log_dups_size":0,"ondisk_log_size":8506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34816000,"num_objects":8500,"num_object_clones":0,"num_object_copies":17000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5671,"num_write_kb":22684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4924},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8455","reported_seq":8577,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:39.899162+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:39.899162+0000","last_peered":"2024-09-11T05:16:39.899162+0000","last_clean":"2024-09-11T05:16:39.899162+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:39.899162+0000","last_undegraded":"2024-09-11T05:16:39.899162+0000","last_fullsized":"2024-09-11T05:16:39.899162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8455,"log_dups_size":0,"ondisk_log_size":8455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34623488,"num_objects":8453,"num_object_clones":0,"num_object_copies":16906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5758,"num_write_kb":23032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8551","reported_seq":8682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:40.844872+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:40.844872+0000","last_peered":"2024-09-11T05:16:40.844872+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:40.844872+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8551,"log_dups_size":0,"ondisk_log_size":8551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34996224,"num_objects":8544,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6619,"num_write_kb":26476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5019}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8378","reported_seq":8543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.872890+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:39.872890+0000","last_peered":"2024-09-11T05:16:39.872890+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:39.872890+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34283520,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4928},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8537","reported_seq":8654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:40.818442+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:40.818442+0000","last_peered":"2024-09-11T05:16:40.818442+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:40.818442+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8537,"log_dups_size":0,"ondisk_log_size":8537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34947072,"num_objects":8532,"num_object_clones":0,"num_object_copies":17064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5010}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8402","reported_seq":15487,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:40.867680+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:40.867680+0000","last_peered":"2024-09-11T05:16:40.867680+0000","last_clean":"2024-09-11T05:16:40.867680+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:40.867680+0000","last_undegraded":"2024-09-11T05:16:40.867680+0000","last_fullsized":"2024-09-11T05:16:40.867680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8402,"log_dups_size":0,"ondisk_log_size":8402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34390016,"num_objects":8396,"num_object_clones":0,"num_object_copies":16792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6305,"num_write_kb":25220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8636","reported_seq":8766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.845556+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:39.845556+0000","last_peered":"2024-09-11T05:16:39.845556+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:39.845556+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8636,"log_dups_size":0,"ondisk_log_size":8636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5669,"num_write_kb":22676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5011},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8334","reported_seq":8467,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.750826+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:39.750826+0000","last_peered":"2024-09-11T05:16:39.750826+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:39.750826+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34103296,"num_objects":8326,"num_object_clones":0,"num_object_copies":16652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5200,"num_write_kb":20800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4831},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8422","reported_seq":8555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.696011+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:39.696011+0000","last_peered":"2024-09-11T05:16:39.696011+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:39.696011+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8422,"log_dups_size":0,"ondisk_log_size":8422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34480128,"num_objects":8418,"num_object_clones":0,"num_object_copies":16836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5180,"num_write_kb":20720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4874},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8388","reported_seq":8531,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.891381+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:39.891381+0000","last_peered":"2024-09-11T05:16:39.891381+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:39.891381+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8388,"log_dups_size":0,"ondisk_log_size":8388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34308713,"num_objects":8377,"num_object_clones":0,"num_object_copies":16754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":555987561,"num_objects":135740,"num_object_clones":0,"num_object_copies":271480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2985,"num_objects_unfound":0,"num_objects_dirty":135740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98885,"num_write_kb":395540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4094,"num_bytes_recovered":16760832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1123966976,"data_stored":1123886959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135821,"ondisk_log_size":135821,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615996,"kb_used_data":385772,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93755844,"statfs":{"total":96636764160,"available":96005984256,"internally_reserved":0,"allocated":395030528,"data_stored":394775674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445404,"kb_used_data":247116,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93926436,"statfs":{"total":96636764160,"available":96180670464,"internally_reserved":0,"allocated":253046784,"data_stored":252718273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764964,"kb_used_data":462612,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93606876,"statfs":{"total":96636764160,"available":95853441024,"internally_reserved":0,"allocated":473714688,"data_stored":473455757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,2,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474484736,"data_stored":474481257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":254083072,"data_stored":254010013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395399168,"data_stored":395395689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:41.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:41.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8583", 'reported_seq': 8661, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.843679+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:39.843679+0000', 'last_peered': '2024-09-11T05:16:39.843679+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:39.843679+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 8583, 'log_dups_size': 0, 'ondisk_log_size': 8583, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35155968, 'num_objects': 8583, 'num_object_clones': 0, 'num_object_copies': 17166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8583, 'num_write_kb': 34332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4908}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8399", 'reported_seq': 8524, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:40.875389+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:40.875389+0000', 'last_peered': '2024-09-11T05:16:40.875389+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:40.875389+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 8399, 'log_dups_size': 0, 'ondisk_log_size': 8399, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34385920, 'num_objects': 8395, 'num_object_clones': 0, 'num_object_copies': 16790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6802, 'num_write_kb': 27208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 4901}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8532", 'reported_seq': 9908, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.891322+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:39.891322+0000', 'last_peered': '2024-09-11T05:16:39.891322+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:39.891322+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 8532, 'log_dups_size': 0, 'ondisk_log_size': 8532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34926592, 'num_objects': 8527, 'num_object_clones': 0, 'num_object_copies': 17054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2985, 'num_objects_unfound': 0, 'num_objects_dirty': 8527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5076, 'num_write_kb': 20304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 613, 'num_bytes_recovered': 2510848, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5540}, {'shards': '0,2', 'objects': 2985}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8506", 'reported_seq': 8638, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.915322+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:39.915322+0000', 'last_peered': '2024-09-11T05:16:39.915322+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:39.915322+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 8506, 'log_dups_size': 0, 'ondisk_log_size': 8506, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34816000, 'num_objects': 8500, 'num_object_clones': 0, 'num_object_copies': 17000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5671, 'num_write_kb': 22684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4924}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8551", 'reported_seq': 8682, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:40.844872+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:40.844872+0000', 'last_peered': '2024-09-11T05:16:40.844872+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:40.844872+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 8551, 'log_dups_size': 0, 'ondisk_log_size': 8551, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34996224, 'num_objects': 8544, 'num_object_clones': 0, 'num_object_copies': 17088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6619, 'num_write_kb': 26476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5019}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8378", 'reported_seq': 8543, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.872890+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:39.872890+0000', 'last_peered': '2024-09-11T05:16:39.872890+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:39.872890+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 8378, 'log_dups_size': 0, 'ondisk_log_size': 8378, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34283520, 'num_objects': 8370, 'num_object_clones': 0, 'num_object_copies': 16740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5131, 'num_write_kb': 20524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4928}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8537", 'reported_seq': 8654, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:40.818442+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:40.818442+0000', 'last_peered': '2024-09-11T05:16:40.818442+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:40.818442+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 8537, 'log_dups_size': 0, 'ondisk_log_size': 8537, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34947072, 'num_objects': 8532, 'num_object_clones': 0, 'num_object_copies': 17064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8532, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6566, 'num_write_kb': 26264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 5010}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8636", 'reported_seq': 8766, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.845556+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:39.845556+0000', 'last_peered': '2024-09-11T05:16:39.845556+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:39.845556+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 8636, 'log_dups_size': 0, 'ondisk_log_size': 8636, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35364864, 'num_objects': 8634, 'num_object_clones': 0, 'num_object_copies': 17268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5669, 'num_write_kb': 22676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5011}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8334", 'reported_seq': 8467, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.750826+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:39.750826+0000', 'last_peered': '2024-09-11T05:16:39.750826+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:39.750826+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 8334, 'log_dups_size': 0, 'ondisk_log_size': 8334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34103296, 'num_objects': 8326, 'num_object_clones': 0, 'num_object_copies': 16652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5200, 'num_write_kb': 20800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4831}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8422", 'reported_seq': 8555, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.696011+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:39.696011+0000', 'last_peered': '2024-09-11T05:16:39.696011+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:39.696011+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 8422, 'log_dups_size': 0, 'ondisk_log_size': 8422, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34480128, 'num_objects': 8418, 'num_object_clones': 0, 'num_object_copies': 16836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5180, 'num_write_kb': 20720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4874}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:41.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8388", 'reported_seq': 8531, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:39.891381+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:39.891381+0000', 'last_peered': '2024-09-11T05:16:39.891381+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:39.891381+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 8388, 'log_dups_size': 0, 'ondisk_log_size': 8388, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34308713, 'num_objects': 8377, 'num_object_clones': 0, 'num_object_copies': 16754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5395, 'num_write_kb': 21580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4862}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:41.651 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:42.142 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 189 20 136590 136570 2.82145 1.71875 0.0238119 0.0276788 2024-09-11T05:16:42.654 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:42.655 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/271484 objects degraded (13.051%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":280,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135742,"data_bytes":555987580,"bytes_used":1870196736,"bytes_avail":288040095744,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":271484,"degraded_ratio":0.13050861192556468,"misplaced_objects":2985,"misplaced_total":271484,"misplaced_ratio":0.010995123101177234,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":67206,"recovering_keys_per_sec":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"write_bytes_sec":2448104,"read_op_per_sec":0,"write_op_per_sec":597},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:42.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:42.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:43.142 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 190 20 137028 137008 2.81561 1.71094 0.0135603 0.0277386 2024-09-11T05:16:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:43.662 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:43.733 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-11T05:16:43.433681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555987580,"num_objects":135742,"num_object_clones":0,"num_object_copies":271484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2985,"num_objects_unfound":0,"num_objects_dirty":135742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98887,"num_write_kb":395542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4095,"num_bytes_recovered":16760851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135824,"ondisk_log_size":135824,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1838012,"kb_used_data":1102988,"kb_used_omap":12,"kb_used_meta":734963,"kb_avail":281277508,"statfs":{"total":289910292480,"available":288028168192,"internally_reserved":0,"allocated":1129459712,"data_stored":1128617416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":752602113},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,4,7,4,8,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19517440,"num_objects":4765,"num_object_clones":0,"num_object_copies":9530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-131,"num_objects_unfound":0,"num_objects_dirty":4765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4765,"num_write_kb":19060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":536576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006464"},"pg_stats":[{"pgid":"2.f","version":"45'8583","reported_seq":8661,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.843679+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:39.843679+0000","last_peered":"2024-09-11T05:16:39.843679+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:39.843679+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8583,"log_dups_size":0,"ondisk_log_size":8583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35155968,"num_objects":8583,"num_object_clones":0,"num_object_copies":17166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":34332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4908},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8542","reported_seq":8644,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:40.971577+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:40.971577+0000","last_peered":"2024-09-11T05:16:40.971577+0000","last_clean":"2024-09-11T05:16:40.971577+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:40.971577+0000","last_undegraded":"2024-09-11T05:16:40.971577+0000","last_fullsized":"2024-09-11T05:16:40.971577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34963456,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6940,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8399","reported_seq":8524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:40.875389+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:40.875389+0000","last_peered":"2024-09-11T05:16:40.875389+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:40.875389+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8399,"log_dups_size":0,"ondisk_log_size":8399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34385920,"num_objects":8395,"num_object_clones":0,"num_object_copies":16790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6802,"num_write_kb":27208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":4901}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8540","reported_seq":8610,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:40.957496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:40.957496+0000","last_peered":"2024-09-11T05:16:40.957496+0000","last_clean":"2024-09-11T05:16:40.957496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:40.957496+0000","last_undegraded":"2024-09-11T05:16:40.957496+0000","last_fullsized":"2024-09-11T05:16:40.957496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34979840,"num_objects":8540,"num_object_clones":0,"num_object_copies":17080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8540,"num_write_kb":34160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8616","reported_seq":8745,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:39.752698+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:39.752698+0000","last_peered":"2024-09-11T05:16:39.752698+0000","last_clean":"2024-09-11T05:16:39.752698+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:39.752698+0000","last_undegraded":"2024-09-11T05:16:39.752698+0000","last_fullsized":"2024-09-11T05:16:39.752698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8616,"log_dups_size":0,"ondisk_log_size":8616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5450,"num_write_kb":21800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8532","reported_seq":9908,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.891322+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:39.891322+0000","last_peered":"2024-09-11T05:16:39.891322+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:39.891322+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8532,"log_dups_size":0,"ondisk_log_size":8532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34926592,"num_objects":8527,"num_object_clones":0,"num_object_copies":17054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2985,"num_objects_unfound":0,"num_objects_dirty":8527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5076,"num_write_kb":20304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":613,"num_bytes_recovered":2510848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5540},{"shards":"0,2","objects":2985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8506","reported_seq":8638,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.915322+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:39.915322+0000","last_peered":"2024-09-11T05:16:39.915322+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:39.915322+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8506,"log_dups_size":0,"ondisk_log_size":8506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34816000,"num_objects":8500,"num_object_clones":0,"num_object_copies":17000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5671,"num_write_kb":22684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4924},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8455","reported_seq":8577,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:39.899162+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:39.899162+0000","last_peered":"2024-09-11T05:16:39.899162+0000","last_clean":"2024-09-11T05:16:39.899162+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:39.899162+0000","last_undegraded":"2024-09-11T05:16:39.899162+0000","last_fullsized":"2024-09-11T05:16:39.899162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8455,"log_dups_size":0,"ondisk_log_size":8455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34623488,"num_objects":8453,"num_object_clones":0,"num_object_copies":16906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5758,"num_write_kb":23032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8551","reported_seq":8682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:40.844872+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:40.844872+0000","last_peered":"2024-09-11T05:16:40.844872+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:40.844872+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8551,"log_dups_size":0,"ondisk_log_size":8551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34996224,"num_objects":8544,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6619,"num_write_kb":26476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5019}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8378","reported_seq":8543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.872890+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:39.872890+0000","last_peered":"2024-09-11T05:16:39.872890+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:39.872890+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34283520,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4928},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8537","reported_seq":8654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:40.818442+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:40.818442+0000","last_peered":"2024-09-11T05:16:40.818442+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:40.818442+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8537,"log_dups_size":0,"ondisk_log_size":8537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34947072,"num_objects":8532,"num_object_clones":0,"num_object_copies":17064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5010}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8402","reported_seq":15487,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:40.867680+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:40.867680+0000","last_peered":"2024-09-11T05:16:40.867680+0000","last_clean":"2024-09-11T05:16:40.867680+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:40.867680+0000","last_undegraded":"2024-09-11T05:16:40.867680+0000","last_fullsized":"2024-09-11T05:16:40.867680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8402,"log_dups_size":0,"ondisk_log_size":8402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34390016,"num_objects":8396,"num_object_clones":0,"num_object_copies":16792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6305,"num_write_kb":25220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8636","reported_seq":8766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.845556+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:39.845556+0000","last_peered":"2024-09-11T05:16:39.845556+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:39.845556+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8636,"log_dups_size":0,"ondisk_log_size":8636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5669,"num_write_kb":22676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5011},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8334","reported_seq":8467,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.750826+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:39.750826+0000","last_peered":"2024-09-11T05:16:39.750826+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:39.750826+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34103296,"num_objects":8326,"num_object_clones":0,"num_object_copies":16652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5200,"num_write_kb":20800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4831},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8422","reported_seq":8555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.696011+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:39.696011+0000","last_peered":"2024-09-11T05:16:39.696011+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:39.696011+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8422,"log_dups_size":0,"ondisk_log_size":8422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34480128,"num_objects":8418,"num_object_clones":0,"num_object_copies":16836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5180,"num_write_kb":20720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4874},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8388","reported_seq":8531,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:39.891381+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:39.891381+0000","last_peered":"2024-09-11T05:16:39.891381+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:39.891381+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8388,"log_dups_size":0,"ondisk_log_size":8388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34308713,"num_objects":8377,"num_object_clones":0,"num_object_copies":16754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":555987561,"num_objects":135740,"num_object_clones":0,"num_object_copies":271480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2985,"num_objects_unfound":0,"num_objects_dirty":135740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98885,"num_write_kb":395540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4094,"num_bytes_recovered":16760832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1131544576,"data_stored":1131464559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135821,"ondisk_log_size":135821,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615996,"kb_used_data":385772,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93755844,"statfs":{"total":96636764160,"available":96005984256,"internally_reserved":0,"allocated":395030528,"data_stored":394775674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457052,"kb_used_data":254604,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93914788,"statfs":{"total":96636764160,"available":96168742912,"internally_reserved":0,"allocated":260714496,"data_stored":260385985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":207285931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764964,"kb_used_data":462612,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93606876,"statfs":{"total":96636764160,"available":95853441024,"internally_reserved":0,"allocated":473714688,"data_stored":473455757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,2,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474484736,"data_stored":474481257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261660672,"data_stored":261587613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395399168,"data_stored":395395689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:44.142 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 191 20 137496 137476 2.81044 1.82812 0.00582622 0.0277889 2024-09-11T05:16:45.143 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 192 20 137976 137956 2.80557 1.875 0.0342028 0.0278389 2024-09-11T05:16:46.146 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 193 20 138461 138441 2.8008 1.89453 0.0657959 0.0278867 2024-09-11T05:16:46.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:47.146 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 194 20 138921 138901 2.79562 1.79688 0.0798165 0.0279363 2024-09-11T05:16:47.650 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:47.652 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:47.693 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-11T05:16:47.434747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565596796,"num_objects":138088,"num_object_clones":0,"num_object_copies":276176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":138088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101233,"num_write_kb":404926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4171,"num_bytes_recovered":17072147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138170,"ondisk_log_size":138170,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1860996,"kb_used_data":1121812,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281254524,"statfs":{"total":289910292480,"available":288004632576,"internally_reserved":0,"allocated":1148735488,"data_stored":1147893192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":756861953},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,7,5,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22941696,"num_objects":5601,"num_object_clones":0,"num_object_copies":11202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-140,"num_objects_unfound":0,"num_objects_dirty":5601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5601,"num_write_kb":22404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":573440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003051"},"pg_stats":[{"pgid":"2.f","version":"45'8750","reported_seq":8828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.970797+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:44.970797+0000","last_peered":"2024-09-11T05:16:44.970797+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:44.970797+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8750,"log_dups_size":0,"ondisk_log_size":8750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35840000,"num_objects":8750,"num_object_clones":0,"num_object_copies":17500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8750,"num_write_kb":35000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5075},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8679","reported_seq":8781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.902745+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:45.902745+0000","last_peered":"2024-09-11T05:16:45.902745+0000","last_clean":"2024-09-11T05:16:45.902745+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:45.902745+0000","last_undegraded":"2024-09-11T05:16:45.902745+0000","last_fullsized":"2024-09-11T05:16:45.902745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8679,"log_dups_size":0,"ondisk_log_size":8679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35524608,"num_objects":8673,"num_object_clones":0,"num_object_copies":17346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7077,"num_write_kb":28308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8555","reported_seq":8678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.919774+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:45.919774+0000","last_peered":"2024-09-11T05:16:45.919774+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:45.919774+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8555,"log_dups_size":0,"ondisk_log_size":8555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35024896,"num_objects":8551,"num_object_clones":0,"num_object_copies":17102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8691","reported_seq":8762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.924531+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:45.924531+0000","last_peered":"2024-09-11T05:16:45.924531+0000","last_clean":"2024-09-11T05:16:45.924531+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:45.924531+0000","last_undegraded":"2024-09-11T05:16:45.924531+0000","last_fullsized":"2024-09-11T05:16:45.924531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8691,"log_dups_size":0,"ondisk_log_size":8691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8691,"num_write_kb":34764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8766","reported_seq":8895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:44.929997+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:44.929997+0000","last_peered":"2024-09-11T05:16:44.929997+0000","last_clean":"2024-09-11T05:16:44.929997+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:44.929997+0000","last_undegraded":"2024-09-11T05:16:44.929997+0000","last_fullsized":"2024-09-11T05:16:44.929997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8766,"log_dups_size":0,"ondisk_log_size":8766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5600,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8663","reported_seq":10191,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.932279+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:44.932279+0000","last_peered":"2024-09-11T05:16:44.932279+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:44.932279+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5207,"num_write_kb":20828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":689,"num_bytes_recovered":2822144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":2909}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8686","reported_seq":8815,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.863533+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:44.863533+0000","last_peered":"2024-09-11T05:16:44.863533+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:44.863533+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8686,"log_dups_size":0,"ondisk_log_size":8686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":23404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5104},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8592","reported_seq":8716,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:44.929305+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:44.929305+0000","last_peered":"2024-09-11T05:16:44.929305+0000","last_clean":"2024-09-11T05:16:44.929305+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:44.929305+0000","last_undegraded":"2024-09-11T05:16:44.929305+0000","last_fullsized":"2024-09-11T05:16:44.929305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8592,"log_dups_size":0,"ondisk_log_size":8592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35184640,"num_objects":8590,"num_object_clones":0,"num_object_copies":17180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5895,"num_write_kb":23580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8715","reported_seq":8843,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.950790+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:45.950790+0000","last_peered":"2024-09-11T05:16:45.950790+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:45.950790+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8715,"log_dups_size":0,"ondisk_log_size":8715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35667968,"num_objects":8708,"num_object_clones":0,"num_object_copies":17416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6783,"num_write_kb":27132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5183}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8511","reported_seq":8675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.921338+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:44.921338+0000","last_peered":"2024-09-11T05:16:44.921338+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:44.921338+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5264,"num_write_kb":21056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8668","reported_seq":8785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.945456+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:45.945456+0000","last_peered":"2024-09-11T05:16:45.945456+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:45.945456+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35483648,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6697,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8553","reported_seq":15638,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.895838+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:45.895838+0000","last_peered":"2024-09-11T05:16:45.895838+0000","last_clean":"2024-09-11T05:16:45.895838+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:45.895838+0000","last_undegraded":"2024-09-11T05:16:45.895838+0000","last_fullsized":"2024-09-11T05:16:45.895838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35008512,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8769","reported_seq":8900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.973400+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:44.973400+0000","last_peered":"2024-09-11T05:16:44.973400+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:44.973400+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8769,"log_dups_size":0,"ondisk_log_size":8769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35909632,"num_objects":8767,"num_object_clones":0,"num_object_copies":17534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5802,"num_write_kb":23208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5144},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8485","reported_seq":8618,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.976339+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:44.976339+0000","last_peered":"2024-09-11T05:16:44.976339+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:44.976339+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8485,"log_dups_size":0,"ondisk_log_size":8485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4982},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8545","reported_seq":8680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.883081+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:44.883081+0000","last_peered":"2024-09-11T05:16:44.883081+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:44.883081+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4997},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8539","reported_seq":8682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.920208+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:44.920208+0000","last_peered":"2024-09-11T05:16:44.920208+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:44.920208+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8539,"log_dups_size":0,"ondisk_log_size":8539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34927209,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5013},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565596777,"num_objects":138086,"num_object_clones":0,"num_object_copies":276172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":138086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101231,"num_write_kb":404924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4170,"num_bytes_recovered":17072128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1151078400,"data_stored":1150998383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138167,"ondisk_log_size":138167,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624740,"kb_used_data":390356,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93747100,"statfs":{"total":96636764160,"available":95997030400,"internally_reserved":0,"allocated":399724544,"data_stored":399469690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456076,"kb_used_data":261948,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93915764,"statfs":{"total":96636764160,"available":96169742336,"internally_reserved":0,"allocated":268234752,"data_stored":267906241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215155,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780180,"kb_used_data":469508,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93591660,"statfs":{"total":96636764160,"available":95837859840,"internally_reserved":0,"allocated":480776192,"data_stored":480517261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481595392,"data_stored":481591913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269254656,"data_stored":269181597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400228352,"data_stored":400224873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:47.694 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:48.146 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 195 20 139356 139336 2.79 1.69922 0.0657373 0.0279915 2024-09-11T05:16:48.423 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:48.423 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:48.505 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-11T05:16:47.434747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565596796,"num_objects":138088,"num_object_clones":0,"num_object_copies":276176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":138088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101233,"num_write_kb":404926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4171,"num_bytes_recovered":17072147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138170,"ondisk_log_size":138170,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1860996,"kb_used_data":1121812,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281254524,"statfs":{"total":289910292480,"available":288004632576,"internally_reserved":0,"allocated":1148735488,"data_stored":1147893192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":756861953},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,7,5,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22941696,"num_objects":5601,"num_object_clones":0,"num_object_copies":11202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-140,"num_objects_unfound":0,"num_objects_dirty":5601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5601,"num_write_kb":22404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":573440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003051"},"pg_stats":[{"pgid":"2.f","version":"45'8750","reported_seq":8828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.970797+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:44.970797+0000","last_peered":"2024-09-11T05:16:44.970797+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:44.970797+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8750,"log_dups_size":0,"ondisk_log_size":8750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35840000,"num_objects":8750,"num_object_clones":0,"num_object_copies":17500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8750,"num_write_kb":35000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5075},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8679","reported_seq":8781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.902745+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:45.902745+0000","last_peered":"2024-09-11T05:16:45.902745+0000","last_clean":"2024-09-11T05:16:45.902745+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:45.902745+0000","last_undegraded":"2024-09-11T05:16:45.902745+0000","last_fullsized":"2024-09-11T05:16:45.902745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8679,"log_dups_size":0,"ondisk_log_size":8679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35524608,"num_objects":8673,"num_object_clones":0,"num_object_copies":17346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7077,"num_write_kb":28308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8555","reported_seq":8678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.919774+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:45.919774+0000","last_peered":"2024-09-11T05:16:45.919774+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:45.919774+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8555,"log_dups_size":0,"ondisk_log_size":8555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35024896,"num_objects":8551,"num_object_clones":0,"num_object_copies":17102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8691","reported_seq":8762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.924531+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:45.924531+0000","last_peered":"2024-09-11T05:16:45.924531+0000","last_clean":"2024-09-11T05:16:45.924531+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:45.924531+0000","last_undegraded":"2024-09-11T05:16:45.924531+0000","last_fullsized":"2024-09-11T05:16:45.924531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8691,"log_dups_size":0,"ondisk_log_size":8691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8691,"num_write_kb":34764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8766","reported_seq":8895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:44.929997+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:44.929997+0000","last_peered":"2024-09-11T05:16:44.929997+0000","last_clean":"2024-09-11T05:16:44.929997+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:44.929997+0000","last_undegraded":"2024-09-11T05:16:44.929997+0000","last_fullsized":"2024-09-11T05:16:44.929997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8766,"log_dups_size":0,"ondisk_log_size":8766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5600,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8663","reported_seq":10191,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.932279+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:44.932279+0000","last_peered":"2024-09-11T05:16:44.932279+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:44.932279+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5207,"num_write_kb":20828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":689,"num_bytes_recovered":2822144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":2909}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8686","reported_seq":8815,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.863533+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:44.863533+0000","last_peered":"2024-09-11T05:16:44.863533+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:44.863533+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8686,"log_dups_size":0,"ondisk_log_size":8686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":23404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5104},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8592","reported_seq":8716,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:44.929305+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:44.929305+0000","last_peered":"2024-09-11T05:16:44.929305+0000","last_clean":"2024-09-11T05:16:44.929305+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:44.929305+0000","last_undegraded":"2024-09-11T05:16:44.929305+0000","last_fullsized":"2024-09-11T05:16:44.929305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8592,"log_dups_size":0,"ondisk_log_size":8592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35184640,"num_objects":8590,"num_object_clones":0,"num_object_copies":17180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5895,"num_write_kb":23580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8715","reported_seq":8843,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.950790+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:45.950790+0000","last_peered":"2024-09-11T05:16:45.950790+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:45.950790+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8715,"log_dups_size":0,"ondisk_log_size":8715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35667968,"num_objects":8708,"num_object_clones":0,"num_object_copies":17416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6783,"num_write_kb":27132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5183}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8511","reported_seq":8675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.921338+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:44.921338+0000","last_peered":"2024-09-11T05:16:44.921338+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:44.921338+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5264,"num_write_kb":21056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8668","reported_seq":8785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.945456+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:45.945456+0000","last_peered":"2024-09-11T05:16:45.945456+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:45.945456+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35483648,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6697,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8553","reported_seq":15638,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.895838+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:45.895838+0000","last_peered":"2024-09-11T05:16:45.895838+0000","last_clean":"2024-09-11T05:16:45.895838+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:45.895838+0000","last_undegraded":"2024-09-11T05:16:45.895838+0000","last_fullsized":"2024-09-11T05:16:45.895838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35008512,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8769","reported_seq":8900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.973400+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:44.973400+0000","last_peered":"2024-09-11T05:16:44.973400+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:44.973400+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8769,"log_dups_size":0,"ondisk_log_size":8769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35909632,"num_objects":8767,"num_object_clones":0,"num_object_copies":17534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5802,"num_write_kb":23208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5144},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8485","reported_seq":8618,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.976339+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:44.976339+0000","last_peered":"2024-09-11T05:16:44.976339+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:44.976339+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8485,"log_dups_size":0,"ondisk_log_size":8485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4982},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8545","reported_seq":8680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.883081+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:44.883081+0000","last_peered":"2024-09-11T05:16:44.883081+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:44.883081+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4997},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8539","reported_seq":8682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.920208+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:44.920208+0000","last_peered":"2024-09-11T05:16:44.920208+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:44.920208+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8539,"log_dups_size":0,"ondisk_log_size":8539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34927209,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5013},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565596777,"num_objects":138086,"num_object_clones":0,"num_object_copies":276172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":138086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101231,"num_write_kb":404924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4170,"num_bytes_recovered":17072128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1151078400,"data_stored":1150998383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138167,"ondisk_log_size":138167,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624740,"kb_used_data":390356,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93747100,"statfs":{"total":96636764160,"available":95997030400,"internally_reserved":0,"allocated":399724544,"data_stored":399469690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456076,"kb_used_data":261948,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93915764,"statfs":{"total":96636764160,"available":96169742336,"internally_reserved":0,"allocated":268234752,"data_stored":267906241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215155,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780180,"kb_used_data":469508,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93591660,"statfs":{"total":96636764160,"available":95837859840,"internally_reserved":0,"allocated":480776192,"data_stored":480517261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481595392,"data_stored":481591913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269254656,"data_stored":269181597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400228352,"data_stored":400224873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8750", 'reported_seq': 8828, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.970797+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:44.970797+0000', 'last_peered': '2024-09-11T05:16:44.970797+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:44.970797+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 8750, 'log_dups_size': 0, 'ondisk_log_size': 8750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35840000, 'num_objects': 8750, 'num_object_clones': 0, 'num_object_copies': 17500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8750, 'num_write_kb': 35000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5075}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8555", 'reported_seq': 8678, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:45.919774+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:45.919774+0000', 'last_peered': '2024-09-11T05:16:45.919774+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:45.919774+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 8555, 'log_dups_size': 0, 'ondisk_log_size': 8555, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35024896, 'num_objects': 8551, 'num_object_clones': 0, 'num_object_copies': 17102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6958, 'num_write_kb': 27832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 5057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8663", 'reported_seq': 10191, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.932279+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:44.932279+0000', 'last_peered': '2024-09-11T05:16:44.932279+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:44.932279+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 8663, 'log_dups_size': 0, 'ondisk_log_size': 8663, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35463168, 'num_objects': 8658, 'num_object_clones': 0, 'num_object_copies': 17316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2909, 'num_objects_unfound': 0, 'num_objects_dirty': 8658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5207, 'num_write_kb': 20828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 689, 'num_bytes_recovered': 2822144, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5747}, {'shards': '0,2', 'objects': 2909}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8686", 'reported_seq': 8815, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.863533+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:44.863533+0000', 'last_peered': '2024-09-11T05:16:44.863533+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:44.863533+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 8686, 'log_dups_size': 0, 'ondisk_log_size': 8686, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35553280, 'num_objects': 8680, 'num_object_clones': 0, 'num_object_copies': 17360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5851, 'num_write_kb': 23404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5104}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8715", 'reported_seq': 8843, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:45.950790+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:45.950790+0000', 'last_peered': '2024-09-11T05:16:45.950790+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:45.950790+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 8715, 'log_dups_size': 0, 'ondisk_log_size': 8715, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35667968, 'num_objects': 8708, 'num_object_clones': 0, 'num_object_copies': 17416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6783, 'num_write_kb': 27132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5183}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8511", 'reported_seq': 8675, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.921338+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:44.921338+0000', 'last_peered': '2024-09-11T05:16:44.921338+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:44.921338+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 8511, 'log_dups_size': 0, 'ondisk_log_size': 8511, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34828288, 'num_objects': 8503, 'num_object_clones': 0, 'num_object_copies': 17006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5264, 'num_write_kb': 21056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5061}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8668", 'reported_seq': 8785, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:45.945456+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:45.945456+0000', 'last_peered': '2024-09-11T05:16:45.945456+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:45.945456+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 8668, 'log_dups_size': 0, 'ondisk_log_size': 8668, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35483648, 'num_objects': 8663, 'num_object_clones': 0, 'num_object_copies': 17326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6697, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 5141}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8769", 'reported_seq': 8900, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.973400+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:44.973400+0000', 'last_peered': '2024-09-11T05:16:44.973400+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:44.973400+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 8769, 'log_dups_size': 0, 'ondisk_log_size': 8769, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35909632, 'num_objects': 8767, 'num_object_clones': 0, 'num_object_copies': 17534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5802, 'num_write_kb': 23208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5144}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8485", 'reported_seq': 8618, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.976339+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:44.976339+0000', 'last_peered': '2024-09-11T05:16:44.976339+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:44.976339+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 8485, 'log_dups_size': 0, 'ondisk_log_size': 8485, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34721792, 'num_objects': 8477, 'num_object_clones': 0, 'num_object_copies': 16954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5351, 'num_write_kb': 21404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4982}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:48.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8545", 'reported_seq': 8680, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.883081+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:44.883081+0000', 'last_peered': '2024-09-11T05:16:44.883081+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:44.883081+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 8545, 'log_dups_size': 0, 'ondisk_log_size': 8545, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34983936, 'num_objects': 8541, 'num_object_clones': 0, 'num_object_copies': 17082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5303, 'num_write_kb': 21212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4997}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:48.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8539", 'reported_seq': 8682, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:44.920208+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:44.920208+0000', 'last_peered': '2024-09-11T05:16:44.920208+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:44.920208+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 8539, 'log_dups_size': 0, 'ondisk_log_size': 8539, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34927209, 'num_objects': 8528, 'num_object_clones': 0, 'num_object_copies': 17056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5546, 'num_write_kb': 22184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5013}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:48.509 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:49.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 196 20 139809 139789 2.78479 1.76953 0.0961017 0.0280453 2024-09-11T05:16:49.659 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:49.660 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/276176 objects degraded (12.829%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":287,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138088,"data_bytes":565596796,"bytes_used":1905659904,"bytes_avail":288004632576,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":276176,"degraded_ratio":0.12829137941023117,"misplaced_objects":2909,"misplaced_total":276176,"misplaced_ratio":0.010533138288627541,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":47765,"recovering_keys_per_sec":0,"num_objects_recovered":140,"num_bytes_recovered":573440,"num_keys_recovered":0,"write_bytes_sec":1617187,"read_op_per_sec":0,"write_op_per_sec":394},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:49.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:49.708 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:50.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 197 20 140239 140219 2.77918 1.67969 0.0434292 0.0280982 2024-09-11T05:16:50.637 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:50.692 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-11T05:16:49.437663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565596796,"num_objects":138088,"num_object_clones":0,"num_object_copies":276176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":138088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101233,"num_write_kb":404926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4171,"num_bytes_recovered":17072147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138170,"ondisk_log_size":138170,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1860996,"kb_used_data":1121812,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281254524,"statfs":{"total":289910292480,"available":288004632576,"internally_reserved":0,"allocated":1148735488,"data_stored":1147893192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":756861953},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,7,5,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19415040,"num_objects":4740,"num_object_clones":0,"num_object_copies":9480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-140,"num_objects_unfound":0,"num_objects_dirty":4740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4740,"num_write_kb":18960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":573440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005435"},"pg_stats":[{"pgid":"2.f","version":"45'8750","reported_seq":8828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.970797+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:44.970797+0000","last_peered":"2024-09-11T05:16:44.970797+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:44.970797+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8750,"log_dups_size":0,"ondisk_log_size":8750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35840000,"num_objects":8750,"num_object_clones":0,"num_object_copies":17500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8750,"num_write_kb":35000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5075},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8679","reported_seq":8781,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.902745+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:45.902745+0000","last_peered":"2024-09-11T05:16:45.902745+0000","last_clean":"2024-09-11T05:16:45.902745+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:45.902745+0000","last_undegraded":"2024-09-11T05:16:45.902745+0000","last_fullsized":"2024-09-11T05:16:45.902745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8679,"log_dups_size":0,"ondisk_log_size":8679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35524608,"num_objects":8673,"num_object_clones":0,"num_object_copies":17346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7077,"num_write_kb":28308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8555","reported_seq":8678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.919774+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:45.919774+0000","last_peered":"2024-09-11T05:16:45.919774+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:45.919774+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8555,"log_dups_size":0,"ondisk_log_size":8555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35024896,"num_objects":8551,"num_object_clones":0,"num_object_copies":17102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8691","reported_seq":8762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.924531+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:45.924531+0000","last_peered":"2024-09-11T05:16:45.924531+0000","last_clean":"2024-09-11T05:16:45.924531+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:45.924531+0000","last_undegraded":"2024-09-11T05:16:45.924531+0000","last_fullsized":"2024-09-11T05:16:45.924531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8691,"log_dups_size":0,"ondisk_log_size":8691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8691,"num_write_kb":34764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8766","reported_seq":8895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:44.929997+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:44.929997+0000","last_peered":"2024-09-11T05:16:44.929997+0000","last_clean":"2024-09-11T05:16:44.929997+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:44.929997+0000","last_undegraded":"2024-09-11T05:16:44.929997+0000","last_fullsized":"2024-09-11T05:16:44.929997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8766,"log_dups_size":0,"ondisk_log_size":8766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5600,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8663","reported_seq":10191,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.932279+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:44.932279+0000","last_peered":"2024-09-11T05:16:44.932279+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:44.932279+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5207,"num_write_kb":20828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":689,"num_bytes_recovered":2822144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5747},{"shards":"0,2","objects":2909}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8686","reported_seq":8815,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.863533+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:44.863533+0000","last_peered":"2024-09-11T05:16:44.863533+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:44.863533+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8686,"log_dups_size":0,"ondisk_log_size":8686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":23404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5104},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8592","reported_seq":8716,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:44.929305+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:44.929305+0000","last_peered":"2024-09-11T05:16:44.929305+0000","last_clean":"2024-09-11T05:16:44.929305+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:44.929305+0000","last_undegraded":"2024-09-11T05:16:44.929305+0000","last_fullsized":"2024-09-11T05:16:44.929305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8592,"log_dups_size":0,"ondisk_log_size":8592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35184640,"num_objects":8590,"num_object_clones":0,"num_object_copies":17180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5895,"num_write_kb":23580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8715","reported_seq":8843,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.950790+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:45.950790+0000","last_peered":"2024-09-11T05:16:45.950790+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:45.950790+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8715,"log_dups_size":0,"ondisk_log_size":8715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35667968,"num_objects":8708,"num_object_clones":0,"num_object_copies":17416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6783,"num_write_kb":27132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5183}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8511","reported_seq":8675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.921338+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:44.921338+0000","last_peered":"2024-09-11T05:16:44.921338+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:44.921338+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5264,"num_write_kb":21056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8668","reported_seq":8785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:45.945456+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:45.945456+0000","last_peered":"2024-09-11T05:16:45.945456+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:45.945456+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8668,"log_dups_size":0,"ondisk_log_size":8668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35483648,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6697,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5141}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8553","reported_seq":15638,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:45.895838+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:45.895838+0000","last_peered":"2024-09-11T05:16:45.895838+0000","last_clean":"2024-09-11T05:16:45.895838+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:45.895838+0000","last_undegraded":"2024-09-11T05:16:45.895838+0000","last_fullsized":"2024-09-11T05:16:45.895838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35008512,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8769","reported_seq":8900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.973400+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:44.973400+0000","last_peered":"2024-09-11T05:16:44.973400+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:44.973400+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8769,"log_dups_size":0,"ondisk_log_size":8769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35909632,"num_objects":8767,"num_object_clones":0,"num_object_copies":17534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5802,"num_write_kb":23208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5144},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8485","reported_seq":8618,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.976339+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:44.976339+0000","last_peered":"2024-09-11T05:16:44.976339+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:44.976339+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8485,"log_dups_size":0,"ondisk_log_size":8485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4982},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8545","reported_seq":8680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.883081+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:44.883081+0000","last_peered":"2024-09-11T05:16:44.883081+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:44.883081+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4997},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8539","reported_seq":8682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:44.920208+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:44.920208+0000","last_peered":"2024-09-11T05:16:44.920208+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:44.920208+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8539,"log_dups_size":0,"ondisk_log_size":8539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34927209,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5013},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565596777,"num_objects":138086,"num_object_clones":0,"num_object_copies":276172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":138086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101231,"num_write_kb":404924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4170,"num_bytes_recovered":17072128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1151078400,"data_stored":1150998383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138167,"ondisk_log_size":138167,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624740,"kb_used_data":390356,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93747100,"statfs":{"total":96636764160,"available":95997030400,"internally_reserved":0,"allocated":399724544,"data_stored":399469690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456076,"kb_used_data":261948,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93915764,"statfs":{"total":96636764160,"available":96169742336,"internally_reserved":0,"allocated":268234752,"data_stored":267906241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198766251},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215155,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780180,"kb_used_data":469508,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93591660,"statfs":{"total":96636764160,"available":95837859840,"internally_reserved":0,"allocated":480776192,"data_stored":480517261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481595392,"data_stored":481591913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269254656,"data_stored":269181597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400228352,"data_stored":400224873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:51.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 198 20 140698 140678 2.7742 1.79297 0.0560021 0.0281541 2024-09-11T05:16:52.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 199 20 141168 141148 2.76949 1.83594 0.0469593 0.0282012 2024-09-11T05:16:53.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:16:53.148966+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0282463 2024-09-11T05:16:53.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:16:53.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 200 20 141649 141629 2.76504 1.87891 0.0635741 0.0282463 2024-09-11T05:16:53.694 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:54.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 201 20 142093 142073 2.75991 1.73438 0.0538814 0.0282988 2024-09-11T05:16:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:54.549 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:54.625 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-11T05:16:53.438616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574804604,"num_objects":140336,"num_object_clones":0,"num_object_copies":280672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2823,"num_objects_unfound":0,"num_objects_dirty":140336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103481,"num_write_kb":413918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":17424403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140418,"ondisk_log_size":140418,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1875180,"kb_used_data":1140156,"kb_used_omap":12,"kb_used_meta":734963,"kb_avail":281240340,"statfs":{"total":289910292480,"available":287990108160,"internally_reserved":0,"allocated":1167519744,"data_stored":1166677448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":752602113},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,13,5,6,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18817024,"num_objects":4594,"num_object_clones":0,"num_object_copies":9188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-162,"num_objects_unfound":0,"num_objects_dirty":4594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4594,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005455"},"pg_stats":[{"pgid":"2.f","version":"45'8888","reported_seq":8965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.943558+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:49.943558+0000","last_peered":"2024-09-11T05:16:49.943558+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:49.943558+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8888,"log_dups_size":0,"ondisk_log_size":8888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405248,"num_objects":8888,"num_object_clones":0,"num_object_copies":17776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8888,"num_write_kb":35552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5213},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8841","reported_seq":8942,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:50.976016+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:50.976016+0000","last_peered":"2024-09-11T05:16:50.976016+0000","last_clean":"2024-09-11T05:16:50.976016+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:50.976016+0000","last_undegraded":"2024-09-11T05:16:50.976016+0000","last_fullsized":"2024-09-11T05:16:50.976016+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8841,"log_dups_size":0,"ondisk_log_size":8841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36188160,"num_objects":8835,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8695","reported_seq":8819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.962699+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:50.962699+0000","last_peered":"2024-09-11T05:16:50.962699+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:50.962699+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8695,"log_dups_size":0,"ondisk_log_size":8695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7098,"num_write_kb":28392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5197}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8811","reported_seq":8882,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:50.895640+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:50.895640+0000","last_peered":"2024-09-11T05:16:50.895640+0000","last_clean":"2024-09-11T05:16:50.895640+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:50.895640+0000","last_undegraded":"2024-09-11T05:16:50.895640+0000","last_fullsized":"2024-09-11T05:16:50.895640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8811,"log_dups_size":0,"ondisk_log_size":8811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36089856,"num_objects":8811,"num_object_clones":0,"num_object_copies":17622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8811,"num_write_kb":35244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8900","reported_seq":9029,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:49.954287+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:49.954287+0000","last_peered":"2024-09-11T05:16:49.954287+0000","last_clean":"2024-09-11T05:16:49.954287+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:49.954287+0000","last_undegraded":"2024-09-11T05:16:49.954287+0000","last_fullsized":"2024-09-11T05:16:49.954287+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8900,"log_dups_size":0,"ondisk_log_size":8900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5734,"num_write_kb":22936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8795","reported_seq":10495,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.003215+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:50.003215+0000","last_peered":"2024-09-11T05:16:50.003215+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:50.003215+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8795,"log_dups_size":0,"ondisk_log_size":8795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36003840,"num_objects":8790,"num_object_clones":0,"num_object_copies":17580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2823,"num_objects_unfound":0,"num_objects_dirty":8790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5339,"num_write_kb":21356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":775,"num_bytes_recovered":3174400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5965},{"shards":"0,2","objects":2823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8839","reported_seq":8971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.952170+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:49.952170+0000","last_peered":"2024-09-11T05:16:49.952170+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:49.952170+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8839,"log_dups_size":0,"ondisk_log_size":8839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5257},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8755","reported_seq":8878,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:49.929431+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:49.929431+0000","last_peered":"2024-09-11T05:16:49.929431+0000","last_clean":"2024-09-11T05:16:49.929431+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:49.929431+0000","last_undegraded":"2024-09-11T05:16:49.929431+0000","last_fullsized":"2024-09-11T05:16:49.929431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8755,"log_dups_size":0,"ondisk_log_size":8755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6058,"num_write_kb":24232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8862","reported_seq":8991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.955988+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:50.955988+0000","last_peered":"2024-09-11T05:16:50.955988+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:50.955988+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8862,"log_dups_size":0,"ondisk_log_size":8862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":27720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8635","reported_seq":8800,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.952163+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:49.952163+0000","last_peered":"2024-09-11T05:16:49.952163+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:49.952163+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8635,"log_dups_size":0,"ondisk_log_size":8635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5185},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8812","reported_seq":8929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.976296+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:50.976296+0000","last_peered":"2024-09-11T05:16:50.976296+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:50.976296+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8812,"log_dups_size":0,"ondisk_log_size":8812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36073472,"num_objects":8807,"num_object_clones":0,"num_object_copies":17614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6841,"num_write_kb":27364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5285}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8683","reported_seq":15768,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:50.955639+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:50.955639+0000","last_peered":"2024-09-11T05:16:50.955639+0000","last_clean":"2024-09-11T05:16:50.955639+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:50.955639+0000","last_undegraded":"2024-09-11T05:16:50.955639+0000","last_fullsized":"2024-09-11T05:16:50.955639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8683,"log_dups_size":0,"ondisk_log_size":8683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35540992,"num_objects":8677,"num_object_clones":0,"num_object_copies":17354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8901","reported_seq":9032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.926197+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:49.926197+0000","last_peered":"2024-09-11T05:16:49.926197+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:49.926197+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36450304,"num_objects":8899,"num_object_clones":0,"num_object_copies":17798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5934,"num_write_kb":23736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5276},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8638","reported_seq":8771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.848937+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:49.848937+0000","last_peered":"2024-09-11T05:16:49.848937+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:49.848937+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5504,"num_write_kb":22016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5135},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8696","reported_seq":8831,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.013275+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:50.013275+0000","last_peered":"2024-09-11T05:16:50.013275+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:50.013275+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35602432,"num_objects":8692,"num_object_clones":0,"num_object_copies":17384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5454,"num_write_kb":21816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5148},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8664","reported_seq":8807,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.946951+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:49.946951+0000","last_peered":"2024-09-11T05:16:49.946951+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:49.946951+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8664,"log_dups_size":0,"ondisk_log_size":8664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35439209,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5671,"num_write_kb":22684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5138},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574804585,"num_objects":140334,"num_object_clones":0,"num_object_copies":280668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2823,"num_objects_unfound":0,"num_objects_dirty":140334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103479,"num_write_kb":413916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4256,"num_bytes_recovered":17424384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1169874944,"data_stored":1169794927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140415,"ondisk_log_size":140415,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633540,"kb_used_data":394996,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93738300,"statfs":{"total":96636764160,"available":95988019200,"internally_reserved":0,"allocated":404475904,"data_stored":404221050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467140,"kb_used_data":268852,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93904700,"statfs":{"total":96636764160,"available":96158412800,"internally_reserved":0,"allocated":275304448,"data_stored":274975937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774500,"kb_used_data":476308,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93597340,"statfs":{"total":96636764160,"available":95843676160,"internally_reserved":0,"allocated":487739392,"data_stored":487480461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,4,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488574976,"data_stored":488571497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276418560,"data_stored":276345501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404881408,"data_stored":404877929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:54.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:55.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 202 20 142525 142505 2.7546 1.6875 0.0230119 0.0283531 2024-09-11T05:16:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:55.783 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:55.784 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-11T05:16:53.438616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574804604,"num_objects":140336,"num_object_clones":0,"num_object_copies":280672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2823,"num_objects_unfound":0,"num_objects_dirty":140336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103481,"num_write_kb":413918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4257,"num_bytes_recovered":17424403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140418,"ondisk_log_size":140418,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1875180,"kb_used_data":1140156,"kb_used_omap":12,"kb_used_meta":734963,"kb_avail":281240340,"statfs":{"total":289910292480,"available":287990108160,"internally_reserved":0,"allocated":1167519744,"data_stored":1166677448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":752602113},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,13,5,6,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18817024,"num_objects":4594,"num_object_clones":0,"num_object_copies":9188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-162,"num_objects_unfound":0,"num_objects_dirty":4594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4594,"num_write_kb":18376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005455"},"pg_stats":[{"pgid":"2.f","version":"45'8888","reported_seq":8965,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.943558+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:49.943558+0000","last_peered":"2024-09-11T05:16:49.943558+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:49.943558+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8888,"log_dups_size":0,"ondisk_log_size":8888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405248,"num_objects":8888,"num_object_clones":0,"num_object_copies":17776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8888,"num_write_kb":35552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5213},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8841","reported_seq":8942,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:50.976016+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:50.976016+0000","last_peered":"2024-09-11T05:16:50.976016+0000","last_clean":"2024-09-11T05:16:50.976016+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:50.976016+0000","last_undegraded":"2024-09-11T05:16:50.976016+0000","last_fullsized":"2024-09-11T05:16:50.976016+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8841,"log_dups_size":0,"ondisk_log_size":8841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36188160,"num_objects":8835,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8695","reported_seq":8819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.962699+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:50.962699+0000","last_peered":"2024-09-11T05:16:50.962699+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:50.962699+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8695,"log_dups_size":0,"ondisk_log_size":8695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7098,"num_write_kb":28392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5197}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8811","reported_seq":8882,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:50.895640+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:50.895640+0000","last_peered":"2024-09-11T05:16:50.895640+0000","last_clean":"2024-09-11T05:16:50.895640+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:50.895640+0000","last_undegraded":"2024-09-11T05:16:50.895640+0000","last_fullsized":"2024-09-11T05:16:50.895640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8811,"log_dups_size":0,"ondisk_log_size":8811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36089856,"num_objects":8811,"num_object_clones":0,"num_object_copies":17622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8811,"num_write_kb":35244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8900","reported_seq":9029,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:49.954287+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:49.954287+0000","last_peered":"2024-09-11T05:16:49.954287+0000","last_clean":"2024-09-11T05:16:49.954287+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:49.954287+0000","last_undegraded":"2024-09-11T05:16:49.954287+0000","last_fullsized":"2024-09-11T05:16:49.954287+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":8900,"log_dups_size":0,"ondisk_log_size":8900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5734,"num_write_kb":22936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8795","reported_seq":10495,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.003215+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:50.003215+0000","last_peered":"2024-09-11T05:16:50.003215+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:50.003215+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8795,"log_dups_size":0,"ondisk_log_size":8795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36003840,"num_objects":8790,"num_object_clones":0,"num_object_copies":17580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2823,"num_objects_unfound":0,"num_objects_dirty":8790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5339,"num_write_kb":21356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":775,"num_bytes_recovered":3174400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5965},{"shards":"0,2","objects":2823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8839","reported_seq":8971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.952170+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:49.952170+0000","last_peered":"2024-09-11T05:16:49.952170+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:49.952170+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":8839,"log_dups_size":0,"ondisk_log_size":8839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5257},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8755","reported_seq":8878,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:49.929431+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:49.929431+0000","last_peered":"2024-09-11T05:16:49.929431+0000","last_clean":"2024-09-11T05:16:49.929431+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:49.929431+0000","last_undegraded":"2024-09-11T05:16:49.929431+0000","last_fullsized":"2024-09-11T05:16:49.929431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8755,"log_dups_size":0,"ondisk_log_size":8755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6058,"num_write_kb":24232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8862","reported_seq":8991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.955988+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:50.955988+0000","last_peered":"2024-09-11T05:16:50.955988+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:50.955988+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":8862,"log_dups_size":0,"ondisk_log_size":8862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":27720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8635","reported_seq":8800,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.952163+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:49.952163+0000","last_peered":"2024-09-11T05:16:49.952163+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:49.952163+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8635,"log_dups_size":0,"ondisk_log_size":8635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5185},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8812","reported_seq":8929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.976296+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:50.976296+0000","last_peered":"2024-09-11T05:16:50.976296+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:50.976296+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8812,"log_dups_size":0,"ondisk_log_size":8812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36073472,"num_objects":8807,"num_object_clones":0,"num_object_copies":17614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6841,"num_write_kb":27364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5285}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8683","reported_seq":15768,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:50.955639+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:50.955639+0000","last_peered":"2024-09-11T05:16:50.955639+0000","last_clean":"2024-09-11T05:16:50.955639+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:50.955639+0000","last_undegraded":"2024-09-11T05:16:50.955639+0000","last_fullsized":"2024-09-11T05:16:50.955639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8683,"log_dups_size":0,"ondisk_log_size":8683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35540992,"num_objects":8677,"num_object_clones":0,"num_object_copies":17354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8901","reported_seq":9032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.926197+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:49.926197+0000","last_peered":"2024-09-11T05:16:49.926197+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:49.926197+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36450304,"num_objects":8899,"num_object_clones":0,"num_object_copies":17798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5934,"num_write_kb":23736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5276},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8638","reported_seq":8771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.848937+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:49.848937+0000","last_peered":"2024-09-11T05:16:49.848937+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:49.848937+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5504,"num_write_kb":22016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5135},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8696","reported_seq":8831,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:50.013275+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:50.013275+0000","last_peered":"2024-09-11T05:16:50.013275+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:50.013275+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35602432,"num_objects":8692,"num_object_clones":0,"num_object_copies":17384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5454,"num_write_kb":21816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5148},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8664","reported_seq":8807,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:49.946951+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:49.946951+0000","last_peered":"2024-09-11T05:16:49.946951+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:49.946951+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8664,"log_dups_size":0,"ondisk_log_size":8664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35439209,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5671,"num_write_kb":22684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5138},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574804585,"num_objects":140334,"num_object_clones":0,"num_object_copies":280668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2823,"num_objects_unfound":0,"num_objects_dirty":140334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103479,"num_write_kb":413916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4256,"num_bytes_recovered":17424384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1169874944,"data_stored":1169794927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140415,"ondisk_log_size":140415,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633540,"kb_used_data":394996,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93738300,"statfs":{"total":96636764160,"available":95988019200,"internally_reserved":0,"allocated":404475904,"data_stored":404221050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467140,"kb_used_data":268852,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93904700,"statfs":{"total":96636764160,"available":96158412800,"internally_reserved":0,"allocated":275304448,"data_stored":274975937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":203026091},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215156,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774500,"kb_used_data":476308,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93597340,"statfs":{"total":96636764160,"available":95843676160,"internally_reserved":0,"allocated":487739392,"data_stored":487480461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,4,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488574976,"data_stored":488571497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276418560,"data_stored":276345501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404881408,"data_stored":404877929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:55.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:16:55.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8888", 'reported_seq': 8965, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:49.943558+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:16:49.943558+0000', 'last_peered': '2024-09-11T05:16:49.943558+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:16:49.943558+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 8888, 'log_dups_size': 0, 'ondisk_log_size': 8888, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36405248, 'num_objects': 8888, 'num_object_clones': 0, 'num_object_copies': 17776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8888, 'num_write_kb': 35552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5213}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:16:55.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8695", 'reported_seq': 8819, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:50.962699+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:16:50.962699+0000', 'last_peered': '2024-09-11T05:16:50.962699+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:16:50.962699+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 8695, 'log_dups_size': 0, 'ondisk_log_size': 8695, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35598336, 'num_objects': 8691, 'num_object_clones': 0, 'num_object_copies': 17382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7098, 'num_write_kb': 28392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 5197}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:55.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:16:55.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8795", 'reported_seq': 10495, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:50.003215+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:16:50.003215+0000', 'last_peered': '2024-09-11T05:16:50.003215+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:16:50.003215+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 8795, 'log_dups_size': 0, 'ondisk_log_size': 8795, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36003840, 'num_objects': 8790, 'num_object_clones': 0, 'num_object_copies': 17580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2823, 'num_objects_unfound': 0, 'num_objects_dirty': 8790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5339, 'num_write_kb': 21356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 775, 'num_bytes_recovered': 3174400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5965}, {'shards': '0,2', 'objects': 2823}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8839", 'reported_seq': 8971, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:49.952170+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:16:49.952170+0000', 'last_peered': '2024-09-11T05:16:49.952170+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:16:49.952170+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 8839, 'log_dups_size': 0, 'ondisk_log_size': 8839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36179968, 'num_objects': 8833, 'num_object_clones': 0, 'num_object_copies': 17666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6004, 'num_write_kb': 24016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5257}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8862", 'reported_seq': 8991, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:50.955988+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:16:50.955988+0000', 'last_peered': '2024-09-11T05:16:50.955988+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:16:50.955988+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 8862, 'log_dups_size': 0, 'ondisk_log_size': 8862, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36270080, 'num_objects': 8855, 'num_object_clones': 0, 'num_object_copies': 17710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8855, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6930, 'num_write_kb': 27720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5330}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8635", 'reported_seq': 8800, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:49.952163+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:16:49.952163+0000', 'last_peered': '2024-09-11T05:16:49.952163+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:16:49.952163+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 8635, 'log_dups_size': 0, 'ondisk_log_size': 8635, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35336192, 'num_objects': 8627, 'num_object_clones': 0, 'num_object_copies': 17254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5388, 'num_write_kb': 21552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5185}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8812", 'reported_seq': 8929, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:50.976296+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:16:50.976296+0000', 'last_peered': '2024-09-11T05:16:50.976296+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:16:50.976296+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 8812, 'log_dups_size': 0, 'ondisk_log_size': 8812, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36073472, 'num_objects': 8807, 'num_object_clones': 0, 'num_object_copies': 17614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6841, 'num_write_kb': 27364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 5285}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:16:55.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8901", 'reported_seq': 9032, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:49.926197+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:16:49.926197+0000', 'last_peered': '2024-09-11T05:16:49.926197+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:16:49.926197+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 8901, 'log_dups_size': 0, 'ondisk_log_size': 8901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36450304, 'num_objects': 8899, 'num_object_clones': 0, 'num_object_copies': 17798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8899, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5934, 'num_write_kb': 23736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5276}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:16:55.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8638", 'reported_seq': 8771, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:49.848937+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:16:49.848937+0000', 'last_peered': '2024-09-11T05:16:49.848937+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:16:49.848937+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 8638, 'log_dups_size': 0, 'ondisk_log_size': 8638, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35348480, 'num_objects': 8630, 'num_object_clones': 0, 'num_object_copies': 17260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5504, 'num_write_kb': 22016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5135}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:16:55.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8696", 'reported_seq': 8831, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:50.013275+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:16:50.013275+0000', 'last_peered': '2024-09-11T05:16:50.013275+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:16:50.013275+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 8696, 'log_dups_size': 0, 'ondisk_log_size': 8696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35602432, 'num_objects': 8692, 'num_object_clones': 0, 'num_object_copies': 17384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5454, 'num_write_kb': 21816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5148}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:16:55.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8664", 'reported_seq': 8807, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:16:49.946951+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:16:49.946951+0000', 'last_peered': '2024-09-11T05:16:49.946951+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:16:49.946951+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 8664, 'log_dups_size': 0, 'ondisk_log_size': 8664, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35439209, 'num_objects': 8653, 'num_object_clones': 0, 'num_object_copies': 17306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8653, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5671, 'num_write_kb': 22684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5138}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:16:55.787 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:16:56.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 203 20 142973 142953 2.74965 1.75 0.0240503 0.0284037 2024-09-11T05:16:56.903 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:56.903 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/283582 objects degraded (12.494%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":294,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":141791,"data_bytes":580764284,"bytes_used":1935855616,"bytes_avail":287974436864,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":283582,"degraded_ratio":0.12494093419187395,"misplaced_objects":2744,"misplaced_total":283582,"misplaced_ratio":0.0096762135819621845,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":82224,"recovering_keys_per_sec":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"write_bytes_sec":2063797,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:16:56.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:16:56.952 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:16:57.147 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 204 20 143419 143399 2.74471 1.74219 0.0217478 0.0284562 2024-09-11T05:16:57.751 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:16:57.753 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:16:57.803 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-11T05:16:57.439580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":584340092,"num_objects":142664,"num_object_clones":0,"num_object_copies":285328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2744,"num_objects_unfound":0,"num_objects_dirty":142664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105809,"num_write_kb":423230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4336,"num_bytes_recovered":17747987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142746,"ondisk_log_size":142746,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1899500,"kb_used_data":1160316,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281216020,"statfs":{"total":289910292480,"available":287965204480,"internally_reserved":0,"allocated":1188163584,"data_stored":1187321288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":756861953},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,3,2,5,10,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22388736,"num_objects":5466,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-165,"num_objects_unfound":0,"num_objects_dirty":5466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5466,"num_write_kb":21864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":675840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005421"},"pg_stats":[{"pgid":"2.f","version":"45'9031","reported_seq":9108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.089357+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:55.089357+0000","last_peered":"2024-09-11T05:16:55.089357+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:55.089357+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9031,"log_dups_size":0,"ondisk_log_size":9031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36990976,"num_objects":9031,"num_object_clones":0,"num_object_copies":18062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9031,"num_write_kb":36124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5356},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8996","reported_seq":9098,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:56.021625+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:56.021625+0000","last_peered":"2024-09-11T05:16:56.021625+0000","last_clean":"2024-09-11T05:16:56.021625+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:56.021625+0000","last_undegraded":"2024-09-11T05:16:56.021625+0000","last_fullsized":"2024-09-11T05:16:56.021625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8996,"log_dups_size":0,"ondisk_log_size":8996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7394,"num_write_kb":29576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8843","reported_seq":8968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.997461+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:55.997461+0000","last_peered":"2024-09-11T05:16:55.997461+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:55.997461+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7246,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8933","reported_seq":9004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.900463+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:55.900463+0000","last_peered":"2024-09-11T05:16:55.900463+0000","last_clean":"2024-09-11T05:16:55.900463+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:55.900463+0000","last_undegraded":"2024-09-11T05:16:55.900463+0000","last_fullsized":"2024-09-11T05:16:55.900463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8933,"log_dups_size":0,"ondisk_log_size":8933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36589568,"num_objects":8933,"num_object_clones":0,"num_object_copies":17866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8933,"num_write_kb":35732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9032","reported_seq":9161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.048527+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:55.048527+0000","last_peered":"2024-09-11T05:16:55.048527+0000","last_clean":"2024-09-11T05:16:55.048527+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:55.048527+0000","last_undegraded":"2024-09-11T05:16:55.048527+0000","last_fullsized":"2024-09-11T05:16:55.048527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9032,"log_dups_size":0,"ondisk_log_size":9032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966400,"num_objects":9025,"num_object_clones":0,"num_object_copies":18050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5866,"num_write_kb":23464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8935","reported_seq":10793,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.124265+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:55.124265+0000","last_peered":"2024-09-11T05:16:55.124265+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:55.124265+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8935,"log_dups_size":0,"ondisk_log_size":8935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2744,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5479,"num_write_kb":21916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":854,"num_bytes_recovered":3497984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6184},{"shards":"0,2","objects":2744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9008","reported_seq":9140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.075609+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:55.075609+0000","last_peered":"2024-09-11T05:16:55.075609+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:55.075609+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9008,"log_dups_size":0,"ondisk_log_size":9008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6173,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8913","reported_seq":9037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.100790+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:55.100790+0000","last_peered":"2024-09-11T05:16:55.100790+0000","last_clean":"2024-09-11T05:16:55.100790+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:55.100790+0000","last_undegraded":"2024-09-11T05:16:55.100790+0000","last_fullsized":"2024-09-11T05:16:55.100790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8913,"log_dups_size":0,"ondisk_log_size":8913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9016","reported_seq":9146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.859858+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:55.859858+0000","last_peered":"2024-09-11T05:16:55.859858+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:55.859858+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9016,"log_dups_size":0,"ondisk_log_size":9016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36900864,"num_objects":9009,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7084,"num_write_kb":28336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5484}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8787","reported_seq":8952,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.087447+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:55.087447+0000","last_peered":"2024-09-11T05:16:55.087447+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:55.087447+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8787,"log_dups_size":0,"ondisk_log_size":8787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5337},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8986","reported_seq":9100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.954040+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:55.954040+0000","last_peered":"2024-09-11T05:16:55.954040+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:55.954040+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8986,"log_dups_size":0,"ondisk_log_size":8986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36786176,"num_objects":8981,"num_object_clones":0,"num_object_copies":17962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7015,"num_write_kb":28060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5459}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8803","reported_seq":15888,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.885911+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:55.885911+0000","last_peered":"2024-09-11T05:16:55.885911+0000","last_clean":"2024-09-11T05:16:55.885911+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:55.885911+0000","last_undegraded":"2024-09-11T05:16:55.885911+0000","last_fullsized":"2024-09-11T05:16:55.885911+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8803,"log_dups_size":0,"ondisk_log_size":8803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032512,"num_objects":8797,"num_object_clones":0,"num_object_copies":17594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6706,"num_write_kb":26824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9040","reported_seq":9171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.027285+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:55.027285+0000","last_peered":"2024-09-11T05:16:55.027285+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:55.027285+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37019648,"num_objects":9038,"num_object_clones":0,"num_object_copies":18076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6073,"num_write_kb":24292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5415},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8788","reported_seq":8921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.090241+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:55.090241+0000","last_peered":"2024-09-11T05:16:55.090241+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:55.090241+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8788,"log_dups_size":0,"ondisk_log_size":8788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5654,"num_write_kb":22616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8837","reported_seq":8971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.100760+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:55.100760+0000","last_peered":"2024-09-11T05:16:55.100760+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:55.100760+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8837,"log_dups_size":0,"ondisk_log_size":8837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5595,"num_write_kb":22380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5289},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8795","reported_seq":8936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.137999+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:55.137999+0000","last_peered":"2024-09-11T05:16:55.137999+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:55.137999+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8795,"log_dups_size":0,"ondisk_log_size":8795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35975785,"num_objects":8784,"num_object_clones":0,"num_object_copies":17568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5802,"num_write_kb":23208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5269},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":584340073,"num_objects":142662,"num_object_clones":0,"num_object_copies":285324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2744,"num_objects_unfound":0,"num_objects_dirty":142662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105807,"num_write_kb":423228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4335,"num_bytes_recovered":17747968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1189036032,"data_stored":1188956015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142743,"ondisk_log_size":142743,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642252,"kb_used_data":399548,"kb_used_omap":4,"kb_used_meta":242683,"kb_avail":93729588,"statfs":{"total":96636764160,"available":95979098112,"internally_reserved":0,"allocated":409137152,"data_stored":408882298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":248508075},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467444,"kb_used_data":277476,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93904396,"statfs":{"total":96636764160,"available":96158101504,"internally_reserved":0,"allocated":284135424,"data_stored":283806913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789804,"kb_used_data":483292,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93582036,"statfs":{"total":96636764160,"available":95828004864,"internally_reserved":0,"allocated":494891008,"data_stored":494632077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,0,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495661056,"data_stored":495657577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283807744,"data_stored":283734685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409567232,"data_stored":409563753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:16:58.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 205 19 143866 143847 2.73986 1.75 0.0164605 0.028506 2024-09-11T05:16:59.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 206 20 144331 144311 2.73536 1.8125 0.0714468 0.0285539 2024-09-11T05:17:00.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 207 20 144801 144781 2.73102 1.83594 0.0112016 0.0285991 2024-09-11T05:17:00.805 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:01.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 208 20 145242 145222 2.72617 1.72266 0.175048 0.0286488 2024-09-11T05:17:01.439 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:01.441 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:01.512 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-11T05:16:59.441897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":584340092,"num_objects":142664,"num_object_clones":0,"num_object_copies":285328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2744,"num_objects_unfound":0,"num_objects_dirty":142664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105809,"num_write_kb":423230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4336,"num_bytes_recovered":17747987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142746,"ondisk_log_size":142746,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1899500,"kb_used_data":1160316,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281216020,"statfs":{"total":289910292480,"available":287965204480,"internally_reserved":0,"allocated":1188163584,"data_stored":1187321288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":756861953},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,3,2,5,10,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18743296,"num_objects":4576,"num_object_clones":0,"num_object_copies":9152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-165,"num_objects_unfound":0,"num_objects_dirty":4576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4576,"num_write_kb":18304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":675840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007150"},"pg_stats":[{"pgid":"2.f","version":"45'9031","reported_seq":9108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.089357+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:16:55.089357+0000","last_peered":"2024-09-11T05:16:55.089357+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:16:55.089357+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9031,"log_dups_size":0,"ondisk_log_size":9031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36990976,"num_objects":9031,"num_object_clones":0,"num_object_copies":18062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9031,"num_write_kb":36124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5356},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8996","reported_seq":9098,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:56.021625+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:16:56.021625+0000","last_peered":"2024-09-11T05:16:56.021625+0000","last_clean":"2024-09-11T05:16:56.021625+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:16:56.021625+0000","last_undegraded":"2024-09-11T05:16:56.021625+0000","last_fullsized":"2024-09-11T05:16:56.021625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":8996,"log_dups_size":0,"ondisk_log_size":8996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36823040,"num_objects":8990,"num_object_clones":0,"num_object_copies":17980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7394,"num_write_kb":29576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8843","reported_seq":8968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.997461+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:16:55.997461+0000","last_peered":"2024-09-11T05:16:55.997461+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:16:55.997461+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7246,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8933","reported_seq":9004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.900463+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:16:55.900463+0000","last_peered":"2024-09-11T05:16:55.900463+0000","last_clean":"2024-09-11T05:16:55.900463+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:16:55.900463+0000","last_undegraded":"2024-09-11T05:16:55.900463+0000","last_fullsized":"2024-09-11T05:16:55.900463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":8933,"log_dups_size":0,"ondisk_log_size":8933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36589568,"num_objects":8933,"num_object_clones":0,"num_object_copies":17866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8933,"num_write_kb":35732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9032","reported_seq":9161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.048527+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:16:55.048527+0000","last_peered":"2024-09-11T05:16:55.048527+0000","last_clean":"2024-09-11T05:16:55.048527+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:16:55.048527+0000","last_undegraded":"2024-09-11T05:16:55.048527+0000","last_fullsized":"2024-09-11T05:16:55.048527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9032,"log_dups_size":0,"ondisk_log_size":9032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966400,"num_objects":9025,"num_object_clones":0,"num_object_copies":18050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5866,"num_write_kb":23464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8935","reported_seq":10793,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.124265+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:16:55.124265+0000","last_peered":"2024-09-11T05:16:55.124265+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:16:55.124265+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":8935,"log_dups_size":0,"ondisk_log_size":8935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2744,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5479,"num_write_kb":21916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":854,"num_bytes_recovered":3497984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6184},{"shards":"0,2","objects":2744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9008","reported_seq":9140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.075609+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:16:55.075609+0000","last_peered":"2024-09-11T05:16:55.075609+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:16:55.075609+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9008,"log_dups_size":0,"ondisk_log_size":9008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6173,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5426},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8913","reported_seq":9037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.100790+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:16:55.100790+0000","last_peered":"2024-09-11T05:16:55.100790+0000","last_clean":"2024-09-11T05:16:55.100790+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:16:55.100790+0000","last_undegraded":"2024-09-11T05:16:55.100790+0000","last_fullsized":"2024-09-11T05:16:55.100790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":8913,"log_dups_size":0,"ondisk_log_size":8913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9016","reported_seq":9146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.859858+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:16:55.859858+0000","last_peered":"2024-09-11T05:16:55.859858+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:16:55.859858+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9016,"log_dups_size":0,"ondisk_log_size":9016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36900864,"num_objects":9009,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7084,"num_write_kb":28336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5484}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8787","reported_seq":8952,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.087447+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:16:55.087447+0000","last_peered":"2024-09-11T05:16:55.087447+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:16:55.087447+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8787,"log_dups_size":0,"ondisk_log_size":8787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5337},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8986","reported_seq":9100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.954040+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:16:55.954040+0000","last_peered":"2024-09-11T05:16:55.954040+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:16:55.954040+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":8986,"log_dups_size":0,"ondisk_log_size":8986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36786176,"num_objects":8981,"num_object_clones":0,"num_object_copies":17962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7015,"num_write_kb":28060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5459}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8803","reported_seq":15888,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:55.885911+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:16:55.885911+0000","last_peered":"2024-09-11T05:16:55.885911+0000","last_clean":"2024-09-11T05:16:55.885911+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:16:55.885911+0000","last_undegraded":"2024-09-11T05:16:55.885911+0000","last_fullsized":"2024-09-11T05:16:55.885911+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8803,"log_dups_size":0,"ondisk_log_size":8803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36032512,"num_objects":8797,"num_object_clones":0,"num_object_copies":17594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6706,"num_write_kb":26824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9040","reported_seq":9171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.027285+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:16:55.027285+0000","last_peered":"2024-09-11T05:16:55.027285+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:16:55.027285+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37019648,"num_objects":9038,"num_object_clones":0,"num_object_copies":18076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6073,"num_write_kb":24292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5415},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8788","reported_seq":8921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.090241+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:16:55.090241+0000","last_peered":"2024-09-11T05:16:55.090241+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:16:55.090241+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8788,"log_dups_size":0,"ondisk_log_size":8788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5654,"num_write_kb":22616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8837","reported_seq":8971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.100760+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:16:55.100760+0000","last_peered":"2024-09-11T05:16:55.100760+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:16:55.100760+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8837,"log_dups_size":0,"ondisk_log_size":8837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5595,"num_write_kb":22380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5289},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8795","reported_seq":8936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:16:55.137999+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:16:55.137999+0000","last_peered":"2024-09-11T05:16:55.137999+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:16:55.137999+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8795,"log_dups_size":0,"ondisk_log_size":8795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35975785,"num_objects":8784,"num_object_clones":0,"num_object_copies":17568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5802,"num_write_kb":23208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5269},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":584340073,"num_objects":142662,"num_object_clones":0,"num_object_copies":285324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2744,"num_objects_unfound":0,"num_objects_dirty":142662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105807,"num_write_kb":423228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4335,"num_bytes_recovered":17747968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1189036032,"data_stored":1188956015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142743,"ondisk_log_size":142743,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642252,"kb_used_data":399548,"kb_used_omap":4,"kb_used_meta":242683,"kb_avail":93729588,"statfs":{"total":96636764160,"available":95979098112,"internally_reserved":0,"allocated":409137152,"data_stored":408882298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":248508075},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467444,"kb_used_data":277476,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93904396,"statfs":{"total":96636764160,"available":96158101504,"internally_reserved":0,"allocated":284135424,"data_stored":283806913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789804,"kb_used_data":483292,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93582036,"statfs":{"total":96636764160,"available":95828004864,"internally_reserved":0,"allocated":494891008,"data_stored":494632077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,0,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495661056,"data_stored":495657577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283807744,"data_stored":283734685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409567232,"data_stored":409563753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:01.514 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:02.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 209 20 145654 145634 2.72083 1.60938 0.0145411 0.0287042 2024-09-11T05:17:02.208 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:02.208 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:02.267 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-11T05:17:01.442382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593691260,"num_objects":144947,"num_object_clones":0,"num_object_copies":289894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2674,"num_objects_unfound":0,"num_objects_dirty":144947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108092,"num_write_kb":432362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4406,"num_bytes_recovered":18034707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145029,"ondisk_log_size":145029,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1921300,"kb_used_data":1172384,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281194220,"statfs":{"total":289910292480,"available":287942881280,"internally_reserved":0,"allocated":1200521216,"data_stored":1199678920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":766823425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,6,0,8,8,15,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28094464,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6859,"num_write_kb":27436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004718"},"pg_stats":[{"pgid":"2.f","version":"45'9165","reported_seq":9243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.146676+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:00.146676+0000","last_peered":"2024-09-11T05:17:00.146676+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:00.146676+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9165,"log_dups_size":0,"ondisk_log_size":9165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9165,"num_write_kb":36660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5490},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9135","reported_seq":9237,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:01.072261+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:01.072261+0000","last_peered":"2024-09-11T05:17:01.072261+0000","last_clean":"2024-09-11T05:17:01.072261+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:01.072261+0000","last_undegraded":"2024-09-11T05:17:01.072261+0000","last_fullsized":"2024-09-11T05:17:01.072261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9135,"log_dups_size":0,"ondisk_log_size":9135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7533,"num_write_kb":30132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8977","reported_seq":9101,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:01.055460+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:01.055460+0000","last_peered":"2024-09-11T05:17:01.055460+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:01.055460+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7380,"num_write_kb":29520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5479}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9067","reported_seq":9138,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:00.812769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:00.812769+0000","last_peered":"2024-09-11T05:17:00.812769+0000","last_clean":"2024-09-11T05:17:00.812769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:00.812769+0000","last_undegraded":"2024-09-11T05:17:00.812769+0000","last_fullsized":"2024-09-11T05:17:00.812769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9067,"log_dups_size":0,"ondisk_log_size":9067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9067,"num_write_kb":36268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9189","reported_seq":9316,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:00.131239+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:00.131239+0000","last_peered":"2024-09-11T05:17:00.131239+0000","last_clean":"2024-09-11T05:17:00.131239+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:00.131239+0000","last_undegraded":"2024-09-11T05:17:00.131239+0000","last_fullsized":"2024-09-11T05:17:00.131239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9189,"log_dups_size":0,"ondisk_log_size":9189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37609472,"num_objects":9182,"num_object_clones":0,"num_object_copies":18364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6023,"num_write_kb":24092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9074","reported_seq":11072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.161798+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:00.161798+0000","last_peered":"2024-09-11T05:17:00.161798+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:00.161798+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9074,"log_dups_size":0,"ondisk_log_size":9074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146624,"num_objects":9069,"num_object_clones":0,"num_object_copies":18138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2674,"num_objects_unfound":0,"num_objects_dirty":9069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5618,"num_write_kb":22472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":924,"num_bytes_recovered":3784704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6393},{"shards":"0,2","objects":2674}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9137","reported_seq":9269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.044203+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:00.044203+0000","last_peered":"2024-09-11T05:17:00.044203+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:00.044203+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9137,"log_dups_size":0,"ondisk_log_size":9137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6302,"num_write_kb":25208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5555},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9051","reported_seq":9175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:00.101886+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:00.101886+0000","last_peered":"2024-09-11T05:17:00.101886+0000","last_clean":"2024-09-11T05:17:00.101886+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:00.101886+0000","last_undegraded":"2024-09-11T05:17:00.101886+0000","last_fullsized":"2024-09-11T05:17:00.101886+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9051,"log_dups_size":0,"ondisk_log_size":9051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6354,"num_write_kb":25416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9161","reported_seq":9292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:01.019573+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:01.019573+0000","last_peered":"2024-09-11T05:17:01.019573+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:01.019573+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9161,"log_dups_size":0,"ondisk_log_size":9161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":28916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8934","reported_seq":9099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.160794+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:00.160794+0000","last_peered":"2024-09-11T05:17:00.160794+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:00.160794+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8934,"log_dups_size":0,"ondisk_log_size":8934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36560896,"num_objects":8926,"num_object_clones":0,"num_object_copies":17852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5484},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9132","reported_seq":9249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:01.020267+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:01.020267+0000","last_peered":"2024-09-11T05:17:01.020267+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:01.020267+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7161,"num_write_kb":28644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8955","reported_seq":16040,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:01.053724+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:01.053724+0000","last_peered":"2024-09-11T05:17:01.053724+0000","last_clean":"2024-09-11T05:17:01.053724+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:01.053724+0000","last_undegraded":"2024-09-11T05:17:01.053724+0000","last_fullsized":"2024-09-11T05:17:01.053724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8955,"log_dups_size":0,"ondisk_log_size":8955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6858,"num_write_kb":27432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9180","reported_seq":9311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.093053+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:00.093053+0000","last_peered":"2024-09-11T05:17:00.093053+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:00.093053+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9180,"log_dups_size":0,"ondisk_log_size":9180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37593088,"num_objects":9178,"num_object_clones":0,"num_object_copies":18356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6213,"num_write_kb":24852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5555},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8925","reported_seq":9058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.157697+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:00.157697+0000","last_peered":"2024-09-11T05:17:00.157697+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:00.157697+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8925,"log_dups_size":0,"ondisk_log_size":8925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5791,"num_write_kb":23164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5422},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8998","reported_seq":9133,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.143126+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:00.143126+0000","last_peered":"2024-09-11T05:17:00.143126+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:00.143126+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8998,"log_dups_size":0,"ondisk_log_size":8998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36839424,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5756,"num_write_kb":23024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5450},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8946","reported_seq":9088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.161749+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:00.161749+0000","last_peered":"2024-09-11T05:17:00.161749+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:00.161749+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8946,"log_dups_size":0,"ondisk_log_size":8946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36594281,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":23812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5420},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":593691241,"num_objects":144945,"num_object_clones":0,"num_object_copies":289890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2674,"num_objects_unfound":0,"num_objects_dirty":144945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108090,"num_write_kb":432360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4405,"num_bytes_recovered":18034688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200791552,"data_stored":1200711535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145026,"ondisk_log_size":145026,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653188,"kb_used_data":404912,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93718652,"statfs":{"total":96636764160,"available":95967899648,"internally_reserved":0,"allocated":414629888,"data_stored":414375034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467444,"kb_used_data":277476,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93904396,"statfs":{"total":96636764160,"available":96158101504,"internally_reserved":0,"allocated":284135424,"data_stored":283806913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":194506411},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215158,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800668,"kb_used_data":489996,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93571172,"statfs":{"total":96636764160,"available":95816880128,"internally_reserved":0,"allocated":501755904,"data_stored":501496973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502697984,"data_stored":502694505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283807744,"data_stored":283734685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414285824,"data_stored":414282345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9165", 'reported_seq': 9243, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.146676+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:00.146676+0000', 'last_peered': '2024-09-11T05:17:00.146676+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:00.146676+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 9165, 'log_dups_size': 0, 'ondisk_log_size': 9165, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37539840, 'num_objects': 9165, 'num_object_clones': 0, 'num_object_copies': 18330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9165, 'num_write_kb': 36660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5490}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8977", 'reported_seq': 9101, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:01.055460+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:01.055460+0000', 'last_peered': '2024-09-11T05:17:01.055460+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:01.055460+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 8977, 'log_dups_size': 0, 'ondisk_log_size': 8977, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36753408, 'num_objects': 8973, 'num_object_clones': 0, 'num_object_copies': 17946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7380, 'num_write_kb': 29520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 5479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9074", 'reported_seq': 11072, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.161798+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:00.161798+0000', 'last_peered': '2024-09-11T05:17:00.161798+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:00.161798+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 9074, 'log_dups_size': 0, 'ondisk_log_size': 9074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37146624, 'num_objects': 9069, 'num_object_clones': 0, 'num_object_copies': 18138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2674, 'num_objects_unfound': 0, 'num_objects_dirty': 9069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5618, 'num_write_kb': 22472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 924, 'num_bytes_recovered': 3784704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6393}, {'shards': '0,2', 'objects': 2674}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:02.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9137", 'reported_seq': 9269, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.044203+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:00.044203+0000', 'last_peered': '2024-09-11T05:17:00.044203+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:00.044203+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 9137, 'log_dups_size': 0, 'ondisk_log_size': 9137, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37400576, 'num_objects': 9131, 'num_object_clones': 0, 'num_object_copies': 18262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6302, 'num_write_kb': 25208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5555}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9161", 'reported_seq': 9292, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:01.019573+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:01.019573+0000', 'last_peered': '2024-09-11T05:17:01.019573+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:01.019573+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 9161, 'log_dups_size': 0, 'ondisk_log_size': 9161, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37494784, 'num_objects': 9154, 'num_object_clones': 0, 'num_object_copies': 18308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7229, 'num_write_kb': 28916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8934", 'reported_seq': 9099, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.160794+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:00.160794+0000', 'last_peered': '2024-09-11T05:17:00.160794+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:00.160794+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 8934, 'log_dups_size': 0, 'ondisk_log_size': 8934, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36560896, 'num_objects': 8926, 'num_object_clones': 0, 'num_object_copies': 17852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5687, 'num_write_kb': 22748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5484}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9132", 'reported_seq': 9249, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:01.020267+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:01.020267+0000', 'last_peered': '2024-09-11T05:17:01.020267+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:01.020267+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 9132, 'log_dups_size': 0, 'ondisk_log_size': 9132, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37384192, 'num_objects': 9127, 'num_object_clones': 0, 'num_object_copies': 18254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7161, 'num_write_kb': 28644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 5605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9180", 'reported_seq': 9311, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.093053+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:00.093053+0000', 'last_peered': '2024-09-11T05:17:00.093053+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:00.093053+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 9180, 'log_dups_size': 0, 'ondisk_log_size': 9180, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37593088, 'num_objects': 9178, 'num_object_clones': 0, 'num_object_copies': 18356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6213, 'num_write_kb': 24852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5555}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8925", 'reported_seq': 9058, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.157697+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:00.157697+0000', 'last_peered': '2024-09-11T05:17:00.157697+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:00.157697+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 8925, 'log_dups_size': 0, 'ondisk_log_size': 8925, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36524032, 'num_objects': 8917, 'num_object_clones': 0, 'num_object_copies': 17834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8917, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5791, 'num_write_kb': 23164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5422}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8998", 'reported_seq': 9133, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.143126+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:00.143126+0000', 'last_peered': '2024-09-11T05:17:00.143126+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:00.143126+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 8998, 'log_dups_size': 0, 'ondisk_log_size': 8998, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36839424, 'num_objects': 8994, 'num_object_clones': 0, 'num_object_copies': 17988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5756, 'num_write_kb': 23024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5450}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8946", 'reported_seq': 9088, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:00.161749+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:00.161749+0000', 'last_peered': '2024-09-11T05:17:00.161749+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:00.161749+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 8946, 'log_dups_size': 0, 'ondisk_log_size': 8946, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36594281, 'num_objects': 8935, 'num_object_clones': 0, 'num_object_copies': 17870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5953, 'num_write_kb': 23812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5420}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:02.271 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:03.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 210 20 146089 146069 2.71596 1.69922 0.0172272 0.0287564 2024-09-11T05:17:03.357 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:03.357 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/289894 objects degraded (12.222%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":301,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":144947,"data_bytes":593691260,"bytes_used":1967411200,"bytes_avail":287942881280,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":289894,"degraded_ratio":0.12222053578204446,"misplaced_objects":2674,"misplaced_total":289894,"misplaced_ratio":0.0092240612085796869,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":80181,"recovering_keys_per_sec":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"write_bytes_sec":2340285,"read_op_per_sec":0,"write_op_per_sec":571},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:03.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:03.416 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:04.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 211 20 146524 146504 2.71115 1.69922 0.0883736 0.0288082 2024-09-11T05:17:04.217 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:04.217 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:04.281 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-11T05:17:03.442960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593691260,"num_objects":144947,"num_object_clones":0,"num_object_copies":289894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2674,"num_objects_unfound":0,"num_objects_dirty":144947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108092,"num_write_kb":432362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4406,"num_bytes_recovered":18034707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145029,"ondisk_log_size":145029,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1950512,"kb_used_data":1179388,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281165008,"statfs":{"total":289910292480,"available":287912968192,"internally_reserved":0,"allocated":1207693312,"data_stored":1206851016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":789564417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,1,5,10,15,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-149,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":610304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004849"},"pg_stats":[{"pgid":"2.f","version":"45'9165","reported_seq":9243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.146676+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:00.146676+0000","last_peered":"2024-09-11T05:17:00.146676+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:00.146676+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9165,"log_dups_size":0,"ondisk_log_size":9165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9165,"num_write_kb":36660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5490},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9135","reported_seq":9237,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:01.072261+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:01.072261+0000","last_peered":"2024-09-11T05:17:01.072261+0000","last_clean":"2024-09-11T05:17:01.072261+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:01.072261+0000","last_undegraded":"2024-09-11T05:17:01.072261+0000","last_fullsized":"2024-09-11T05:17:01.072261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9135,"log_dups_size":0,"ondisk_log_size":9135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7533,"num_write_kb":30132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8977","reported_seq":9101,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:01.055460+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:01.055460+0000","last_peered":"2024-09-11T05:17:01.055460+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:01.055460+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7380,"num_write_kb":29520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5479}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9067","reported_seq":9138,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:00.812769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:00.812769+0000","last_peered":"2024-09-11T05:17:00.812769+0000","last_clean":"2024-09-11T05:17:00.812769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:00.812769+0000","last_undegraded":"2024-09-11T05:17:00.812769+0000","last_fullsized":"2024-09-11T05:17:00.812769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9067,"log_dups_size":0,"ondisk_log_size":9067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9067,"num_write_kb":36268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9189","reported_seq":9316,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:00.131239+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:00.131239+0000","last_peered":"2024-09-11T05:17:00.131239+0000","last_clean":"2024-09-11T05:17:00.131239+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:00.131239+0000","last_undegraded":"2024-09-11T05:17:00.131239+0000","last_fullsized":"2024-09-11T05:17:00.131239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9189,"log_dups_size":0,"ondisk_log_size":9189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37609472,"num_objects":9182,"num_object_clones":0,"num_object_copies":18364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6023,"num_write_kb":24092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9074","reported_seq":11072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.161798+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:00.161798+0000","last_peered":"2024-09-11T05:17:00.161798+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:00.161798+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9074,"log_dups_size":0,"ondisk_log_size":9074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146624,"num_objects":9069,"num_object_clones":0,"num_object_copies":18138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2674,"num_objects_unfound":0,"num_objects_dirty":9069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5618,"num_write_kb":22472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":924,"num_bytes_recovered":3784704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6393},{"shards":"0,2","objects":2674}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9137","reported_seq":9269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.044203+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:00.044203+0000","last_peered":"2024-09-11T05:17:00.044203+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:00.044203+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9137,"log_dups_size":0,"ondisk_log_size":9137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6302,"num_write_kb":25208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5555},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9051","reported_seq":9175,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:00.101886+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:00.101886+0000","last_peered":"2024-09-11T05:17:00.101886+0000","last_clean":"2024-09-11T05:17:00.101886+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:00.101886+0000","last_undegraded":"2024-09-11T05:17:00.101886+0000","last_fullsized":"2024-09-11T05:17:00.101886+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9051,"log_dups_size":0,"ondisk_log_size":9051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6354,"num_write_kb":25416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9161","reported_seq":9292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:01.019573+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:01.019573+0000","last_peered":"2024-09-11T05:17:01.019573+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:01.019573+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9161,"log_dups_size":0,"ondisk_log_size":9161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":28916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8934","reported_seq":9099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.160794+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:00.160794+0000","last_peered":"2024-09-11T05:17:00.160794+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:00.160794+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":8934,"log_dups_size":0,"ondisk_log_size":8934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36560896,"num_objects":8926,"num_object_clones":0,"num_object_copies":17852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5484},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550159+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550159+0000","last_peered":"2024-09-11T05:15:57.550159+0000","last_clean":"2024-09-11T05:15:57.550159+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550159+0000","last_undegraded":"2024-09-11T05:15:57.550159+0000","last_fullsized":"2024-09-11T05:15:57.550159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9132","reported_seq":9249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:01.020267+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:01.020267+0000","last_peered":"2024-09-11T05:17:01.020267+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:01.020267+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7161,"num_write_kb":28644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8955","reported_seq":16040,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:01.053724+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:01.053724+0000","last_peered":"2024-09-11T05:17:01.053724+0000","last_clean":"2024-09-11T05:17:01.053724+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:01.053724+0000","last_undegraded":"2024-09-11T05:17:01.053724+0000","last_fullsized":"2024-09-11T05:17:01.053724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":8955,"log_dups_size":0,"ondisk_log_size":8955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6858,"num_write_kb":27432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9180","reported_seq":9311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.093053+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:00.093053+0000","last_peered":"2024-09-11T05:17:00.093053+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:00.093053+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9180,"log_dups_size":0,"ondisk_log_size":9180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37593088,"num_objects":9178,"num_object_clones":0,"num_object_copies":18356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6213,"num_write_kb":24852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5555},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8925","reported_seq":9058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.157697+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:00.157697+0000","last_peered":"2024-09-11T05:17:00.157697+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:00.157697+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":8925,"log_dups_size":0,"ondisk_log_size":8925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5791,"num_write_kb":23164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5422},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8998","reported_seq":9133,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.143126+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:00.143126+0000","last_peered":"2024-09-11T05:17:00.143126+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:00.143126+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":8998,"log_dups_size":0,"ondisk_log_size":8998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36839424,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5756,"num_write_kb":23024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5450},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8946","reported_seq":9088,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:00.161749+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:00.161749+0000","last_peered":"2024-09-11T05:17:00.161749+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:00.161749+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":8946,"log_dups_size":0,"ondisk_log_size":8946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36594281,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":23812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5420},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":593691241,"num_objects":144945,"num_object_clones":0,"num_object_copies":289890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2674,"num_objects_unfound":0,"num_objects_dirty":144945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108090,"num_write_kb":432360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4405,"num_bytes_recovered":18034688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1208041472,"data_stored":1207961455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145026,"ondisk_log_size":145026,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653188,"kb_used_data":404912,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93718652,"statfs":{"total":96636764160,"available":95967899648,"internally_reserved":0,"allocated":414629888,"data_stored":414375034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496656,"kb_used_data":284480,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93875184,"statfs":{"total":96636764160,"available":96128188416,"internally_reserved":0,"allocated":291307520,"data_stored":290979009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":217247403},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215158,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800668,"kb_used_data":489996,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93571172,"statfs":{"total":96636764160,"available":95816880128,"internally_reserved":0,"allocated":501755904,"data_stored":501496973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502697984,"data_stored":502694505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":291057664,"data_stored":290984605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414285824,"data_stored":414282345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:05.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 212 20 146983 146963 2.70682 1.79297 0.0428083 0.0288547 2024-09-11T05:17:06.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 213 20 147451 147431 2.70269 1.82812 0.0452374 0.0288981 2024-09-11T05:17:07.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 214 19 147905 147886 2.69837 1.77734 0.0840448 0.0289452 2024-09-11T05:17:07.283 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:08.079 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:08.079 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:08.124 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-11T05:17:07.450780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602845820,"num_objects":147182,"num_object_clones":0,"num_object_copies":294364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":147182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110327,"num_write_kb":441302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4481,"num_bytes_recovered":18341907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147264,"ondisk_log_size":147264,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967768,"kb_used_data":1196648,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281147752,"statfs":{"total":289910292480,"available":287895298048,"internally_reserved":0,"allocated":1225367552,"data_stored":1224525256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":789564417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,3,8,17,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22081536,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-145,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5391,"num_write_kb":21564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":593920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011704"},"pg_stats":[{"pgid":"2.f","version":"45'9305","reported_seq":9382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.213445+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:05.213445+0000","last_peered":"2024-09-11T05:17:05.213445+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:05.213445+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9305,"log_dups_size":0,"ondisk_log_size":9305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38113280,"num_objects":9305,"num_object_clones":0,"num_object_copies":18610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":37220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9270","reported_seq":9372,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:06.031311+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:06.031311+0000","last_peered":"2024-09-11T05:17:06.031311+0000","last_clean":"2024-09-11T05:17:06.031311+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:06.031311+0000","last_undegraded":"2024-09-11T05:17:06.031311+0000","last_fullsized":"2024-09-11T05:17:06.031311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9270,"log_dups_size":0,"ondisk_log_size":9270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37945344,"num_objects":9264,"num_object_clones":0,"num_object_copies":18528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7668,"num_write_kb":30672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9111","reported_seq":9236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:06.026479+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:06.026479+0000","last_peered":"2024-09-11T05:17:06.026479+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:06.026479+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7514,"num_write_kb":30056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9204","reported_seq":9275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.978839+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:05.978839+0000","last_peered":"2024-09-11T05:17:05.978839+0000","last_clean":"2024-09-11T05:17:05.978839+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:05.978839+0000","last_undegraded":"2024-09-11T05:17:05.978839+0000","last_fullsized":"2024-09-11T05:17:05.978839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9204,"num_write_kb":36816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9329","reported_seq":9458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.188261+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:05.188261+0000","last_peered":"2024-09-11T05:17:05.188261+0000","last_clean":"2024-09-11T05:17:05.188261+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:05.188261+0000","last_undegraded":"2024-09-11T05:17:05.188261+0000","last_fullsized":"2024-09-11T05:17:05.188261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38182912,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9204","reported_seq":11352,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.206952+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:05.206952+0000","last_peered":"2024-09-11T05:17:05.206952+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:05.206952+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679104,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":22992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":999,"num_bytes_recovered":4091904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6598},{"shards":"0,2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9291","reported_seq":9423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.130563+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:05.130563+0000","last_peered":"2024-09-11T05:17:05.130563+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:05.130563+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9291,"log_dups_size":0,"ondisk_log_size":9291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38031360,"num_objects":9285,"num_object_clones":0,"num_object_copies":18570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5709},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9186","reported_seq":9310,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.176086+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:05.176086+0000","last_peered":"2024-09-11T05:17:05.176086+0000","last_clean":"2024-09-11T05:17:05.176086+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:05.176086+0000","last_undegraded":"2024-09-11T05:17:05.176086+0000","last_fullsized":"2024-09-11T05:17:05.176086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9319","reported_seq":9450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.998887+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:05.998887+0000","last_peered":"2024-09-11T05:17:05.998887+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:05.998887+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9319,"log_dups_size":0,"ondisk_log_size":9319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7387,"num_write_kb":29548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9063","reported_seq":9228,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.209837+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:05.209837+0000","last_peered":"2024-09-11T05:17:05.209837+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:05.209837+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9063,"log_dups_size":0,"ondisk_log_size":9063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37089280,"num_objects":9055,"num_object_clones":0,"num_object_copies":18110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":23264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9285","reported_seq":9401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:06.004520+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:06.004520+0000","last_peered":"2024-09-11T05:17:06.004520+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:06.004520+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7314,"num_write_kb":29256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5758}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9097","reported_seq":16182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.984791+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:05.984791+0000","last_peered":"2024-09-11T05:17:05.984791+0000","last_clean":"2024-09-11T05:17:05.984791+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:05.984791+0000","last_undegraded":"2024-09-11T05:17:05.984791+0000","last_fullsized":"2024-09-11T05:17:05.984791+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9097,"log_dups_size":0,"ondisk_log_size":9097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37236736,"num_objects":9091,"num_object_clones":0,"num_object_copies":18182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7000,"num_write_kb":28000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9328","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.192713+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:05.192713+0000","last_peered":"2024-09-11T05:17:05.192713+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:05.192713+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5703},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9065","reported_seq":9198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.198140+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:05.198140+0000","last_peered":"2024-09-11T05:17:05.198140+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:05.198140+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9065,"log_dups_size":0,"ondisk_log_size":9065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5931,"num_write_kb":23724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5562},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9121","reported_seq":9256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.159928+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:05.159928+0000","last_peered":"2024-09-11T05:17:05.159928+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:05.159928+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9121,"log_dups_size":0,"ondisk_log_size":9121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9083","reported_seq":9226,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.189156+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:05.189156+0000","last_peered":"2024-09-11T05:17:05.189156+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:05.189156+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9083,"log_dups_size":0,"ondisk_log_size":9083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37155433,"num_objects":9072,"num_object_clones":0,"num_object_copies":18144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6090,"num_write_kb":24360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602845801,"num_objects":147180,"num_object_clones":0,"num_object_copies":294360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":147180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110325,"num_write_kb":441300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4480,"num_bytes_recovered":18341888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226838016,"data_stored":1226757999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147261,"ondisk_log_size":147261,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660944,"kb_used_data":408512,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93710896,"statfs":{"total":96636764160,"available":95959957504,"internally_reserved":0,"allocated":418316288,"data_stored":418061434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512100,"kb_used_data":291604,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93859740,"statfs":{"total":96636764160,"available":96112373760,"internally_reserved":0,"allocated":298602496,"data_stored":298273985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":225767083},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794724,"kb_used_data":496532,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93577116,"statfs":{"total":96636764160,"available":95822966784,"internally_reserved":0,"allocated":508448768,"data_stored":508189837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,3,9,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509440000,"data_stored":509436521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298397696,"data_stored":298324637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419000320,"data_stored":418996841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:08.126 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:08.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 215 20 148343 148323 2.69374 1.70703 0.0281384 0.0289943 2024-09-11T05:17:08.954 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:08.957 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:09.034 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-11T05:17:07.450780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602845820,"num_objects":147182,"num_object_clones":0,"num_object_copies":294364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":147182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110327,"num_write_kb":441302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4481,"num_bytes_recovered":18341907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147264,"ondisk_log_size":147264,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967768,"kb_used_data":1196648,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281147752,"statfs":{"total":289910292480,"available":287895298048,"internally_reserved":0,"allocated":1225367552,"data_stored":1224525256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":789564417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,3,8,17,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22081536,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-145,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5391,"num_write_kb":21564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":593920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011704"},"pg_stats":[{"pgid":"2.f","version":"45'9305","reported_seq":9382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.213445+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:05.213445+0000","last_peered":"2024-09-11T05:17:05.213445+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:05.213445+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9305,"log_dups_size":0,"ondisk_log_size":9305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38113280,"num_objects":9305,"num_object_clones":0,"num_object_copies":18610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":37220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9270","reported_seq":9372,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:06.031311+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:06.031311+0000","last_peered":"2024-09-11T05:17:06.031311+0000","last_clean":"2024-09-11T05:17:06.031311+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:06.031311+0000","last_undegraded":"2024-09-11T05:17:06.031311+0000","last_fullsized":"2024-09-11T05:17:06.031311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9270,"log_dups_size":0,"ondisk_log_size":9270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37945344,"num_objects":9264,"num_object_clones":0,"num_object_copies":18528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7668,"num_write_kb":30672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9111","reported_seq":9236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:06.026479+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:06.026479+0000","last_peered":"2024-09-11T05:17:06.026479+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:06.026479+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7514,"num_write_kb":30056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9204","reported_seq":9275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.978839+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:05.978839+0000","last_peered":"2024-09-11T05:17:05.978839+0000","last_clean":"2024-09-11T05:17:05.978839+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:05.978839+0000","last_undegraded":"2024-09-11T05:17:05.978839+0000","last_fullsized":"2024-09-11T05:17:05.978839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9204,"num_write_kb":36816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9329","reported_seq":9458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.188261+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:05.188261+0000","last_peered":"2024-09-11T05:17:05.188261+0000","last_clean":"2024-09-11T05:17:05.188261+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:05.188261+0000","last_undegraded":"2024-09-11T05:17:05.188261+0000","last_fullsized":"2024-09-11T05:17:05.188261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38182912,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9204","reported_seq":11352,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.206952+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:05.206952+0000","last_peered":"2024-09-11T05:17:05.206952+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:05.206952+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679104,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":22992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":999,"num_bytes_recovered":4091904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6598},{"shards":"0,2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9291","reported_seq":9423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.130563+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:05.130563+0000","last_peered":"2024-09-11T05:17:05.130563+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:05.130563+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9291,"log_dups_size":0,"ondisk_log_size":9291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38031360,"num_objects":9285,"num_object_clones":0,"num_object_copies":18570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5709},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9186","reported_seq":9310,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.176086+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:05.176086+0000","last_peered":"2024-09-11T05:17:05.176086+0000","last_clean":"2024-09-11T05:17:05.176086+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:05.176086+0000","last_undegraded":"2024-09-11T05:17:05.176086+0000","last_fullsized":"2024-09-11T05:17:05.176086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9319","reported_seq":9450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.998887+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:05.998887+0000","last_peered":"2024-09-11T05:17:05.998887+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:05.998887+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9319,"log_dups_size":0,"ondisk_log_size":9319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7387,"num_write_kb":29548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9063","reported_seq":9228,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.209837+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:05.209837+0000","last_peered":"2024-09-11T05:17:05.209837+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:05.209837+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9063,"log_dups_size":0,"ondisk_log_size":9063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37089280,"num_objects":9055,"num_object_clones":0,"num_object_copies":18110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":23264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9285","reported_seq":9401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:06.004520+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:06.004520+0000","last_peered":"2024-09-11T05:17:06.004520+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:06.004520+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7314,"num_write_kb":29256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5758}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9097","reported_seq":16182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.984791+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:05.984791+0000","last_peered":"2024-09-11T05:17:05.984791+0000","last_clean":"2024-09-11T05:17:05.984791+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:05.984791+0000","last_undegraded":"2024-09-11T05:17:05.984791+0000","last_fullsized":"2024-09-11T05:17:05.984791+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9097,"log_dups_size":0,"ondisk_log_size":9097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37236736,"num_objects":9091,"num_object_clones":0,"num_object_copies":18182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7000,"num_write_kb":28000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9328","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.192713+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:05.192713+0000","last_peered":"2024-09-11T05:17:05.192713+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:05.192713+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5703},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9065","reported_seq":9198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.198140+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:05.198140+0000","last_peered":"2024-09-11T05:17:05.198140+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:05.198140+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9065,"log_dups_size":0,"ondisk_log_size":9065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5931,"num_write_kb":23724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5562},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9121","reported_seq":9256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.159928+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:05.159928+0000","last_peered":"2024-09-11T05:17:05.159928+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:05.159928+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9121,"log_dups_size":0,"ondisk_log_size":9121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9083","reported_seq":9226,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.189156+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:05.189156+0000","last_peered":"2024-09-11T05:17:05.189156+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:05.189156+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9083,"log_dups_size":0,"ondisk_log_size":9083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37155433,"num_objects":9072,"num_object_clones":0,"num_object_copies":18144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6090,"num_write_kb":24360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602845801,"num_objects":147180,"num_object_clones":0,"num_object_copies":294360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":147180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110325,"num_write_kb":441300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4480,"num_bytes_recovered":18341888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226838016,"data_stored":1226757999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147261,"ondisk_log_size":147261,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660944,"kb_used_data":408512,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93710896,"statfs":{"total":96636764160,"available":95959957504,"internally_reserved":0,"allocated":418316288,"data_stored":418061434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512100,"kb_used_data":291604,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93859740,"statfs":{"total":96636764160,"available":96112373760,"internally_reserved":0,"allocated":298602496,"data_stored":298273985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":225767083},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794724,"kb_used_data":496532,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93577116,"statfs":{"total":96636764160,"available":95822966784,"internally_reserved":0,"allocated":508448768,"data_stored":508189837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,3,9,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509440000,"data_stored":509436521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298397696,"data_stored":298324637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419000320,"data_stored":418996841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:09.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9305", 'reported_seq': 9382, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.213445+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:05.213445+0000', 'last_peered': '2024-09-11T05:17:05.213445+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:05.213445+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 9305, 'log_dups_size': 0, 'ondisk_log_size': 9305, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38113280, 'num_objects': 9305, 'num_object_clones': 0, 'num_object_copies': 18610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9305, 'num_write_kb': 37220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5630}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9111", 'reported_seq': 9236, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:06.026479+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:06.026479+0000', 'last_peered': '2024-09-11T05:17:06.026479+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:06.026479+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 9111, 'log_dups_size': 0, 'ondisk_log_size': 9111, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37302272, 'num_objects': 9107, 'num_object_clones': 0, 'num_object_copies': 18214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7514, 'num_write_kb': 30056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 5613}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9204", 'reported_seq': 11352, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.206952+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:05.206952+0000', 'last_peered': '2024-09-11T05:17:05.206952+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:05.206952+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 9204, 'log_dups_size': 0, 'ondisk_log_size': 9204, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37679104, 'num_objects': 9199, 'num_object_clones': 0, 'num_object_copies': 18398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2599, 'num_objects_unfound': 0, 'num_objects_dirty': 9199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5748, 'num_write_kb': 22992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 999, 'num_bytes_recovered': 4091904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6598}, {'shards': '0,2', 'objects': 2599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9291", 'reported_seq': 9423, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.130563+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:05.130563+0000', 'last_peered': '2024-09-11T05:17:05.130563+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:05.130563+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 9291, 'log_dups_size': 0, 'ondisk_log_size': 9291, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38031360, 'num_objects': 9285, 'num_object_clones': 0, 'num_object_copies': 18570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6456, 'num_write_kb': 25824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5709}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:09.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9319", 'reported_seq': 9450, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.998887+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:05.998887+0000', 'last_peered': '2024-09-11T05:17:05.998887+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:05.998887+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 9319, 'log_dups_size': 0, 'ondisk_log_size': 9319, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38141952, 'num_objects': 9312, 'num_object_clones': 0, 'num_object_copies': 18624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7387, 'num_write_kb': 29548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5787}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9063", 'reported_seq': 9228, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.209837+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:05.209837+0000', 'last_peered': '2024-09-11T05:17:05.209837+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:05.209837+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 9063, 'log_dups_size': 0, 'ondisk_log_size': 9063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37089280, 'num_objects': 9055, 'num_object_clones': 0, 'num_object_copies': 18110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5816, 'num_write_kb': 23264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5613}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9285", 'reported_seq': 9401, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:06.004520+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:06.004520+0000', 'last_peered': '2024-09-11T05:17:06.004520+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:06.004520+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 9285, 'log_dups_size': 0, 'ondisk_log_size': 9285, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38010880, 'num_objects': 9280, 'num_object_clones': 0, 'num_object_copies': 18560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7314, 'num_write_kb': 29256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 5758}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9328", 'reported_seq': 9459, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.192713+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:05.192713+0000', 'last_peered': '2024-09-11T05:17:05.192713+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:05.192713+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 9328, 'log_dups_size': 0, 'ondisk_log_size': 9328, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38199296, 'num_objects': 9326, 'num_object_clones': 0, 'num_object_copies': 18652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6361, 'num_write_kb': 25444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5703}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9065", 'reported_seq': 9198, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.198140+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:05.198140+0000', 'last_peered': '2024-09-11T05:17:05.198140+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:05.198140+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 9065, 'log_dups_size': 0, 'ondisk_log_size': 9065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37097472, 'num_objects': 9057, 'num_object_clones': 0, 'num_object_copies': 18114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5931, 'num_write_kb': 23724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5562}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:09.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9121", 'reported_seq': 9256, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.159928+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:05.159928+0000', 'last_peered': '2024-09-11T05:17:05.159928+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:05.159928+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 9121, 'log_dups_size': 0, 'ondisk_log_size': 9121, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37343232, 'num_objects': 9117, 'num_object_clones': 0, 'num_object_copies': 18234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5879, 'num_write_kb': 23516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5573}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:09.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9083", 'reported_seq': 9226, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:05.189156+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:05.189156+0000', 'last_peered': '2024-09-11T05:17:05.189156+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:05.189156+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 9083, 'log_dups_size': 0, 'ondisk_log_size': 9083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37155433, 'num_objects': 9072, 'num_object_clones': 0, 'num_object_copies': 18144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6090, 'num_write_kb': 24360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5557}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:09.038 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:09.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 216 20 148786 148766 2.68928 1.73047 0.0195481 0.029042 2024-09-11T05:17:10.117 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:10.117 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/294364 objects degraded (12.036%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":307,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147182,"data_bytes":602845820,"bytes_used":2014994432,"bytes_avail":287895298048,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":294364,"degraded_ratio":0.12036458262559278,"misplaced_objects":2599,"misplaced_total":294364,"misplaced_ratio":0.0088292046581783107,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":49445,"recovering_keys_per_sec":0,"num_objects_recovered":145,"num_bytes_recovered":593920,"num_keys_recovered":0,"write_bytes_sec":1540643,"read_op_per_sec":0,"write_op_per_sec":376},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:10.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 217 20 149226 149206 2.68481 1.71875 0.045391 0.0290908 2024-09-11T05:17:10.220 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:10.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:10.985 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:10.985 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:11.029 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-11T05:17:09.451266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602845820,"num_objects":147182,"num_object_clones":0,"num_object_copies":294364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":147182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110327,"num_write_kb":441302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4481,"num_bytes_recovered":18341907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147264,"ondisk_log_size":147264,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967768,"kb_used_data":1196648,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281147752,"statfs":{"total":289910292480,"available":287895298048,"internally_reserved":0,"allocated":1225367552,"data_stored":1224525256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":789564417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,3,8,17,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18505728,"num_objects":4518,"num_object_clones":0,"num_object_copies":9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-145,"num_objects_unfound":0,"num_objects_dirty":4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4518,"num_write_kb":18072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":593920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011686"},"pg_stats":[{"pgid":"2.f","version":"45'9305","reported_seq":9382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.213445+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:05.213445+0000","last_peered":"2024-09-11T05:17:05.213445+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:05.213445+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9305,"log_dups_size":0,"ondisk_log_size":9305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38113280,"num_objects":9305,"num_object_clones":0,"num_object_copies":18610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":37220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9270","reported_seq":9372,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:06.031311+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:06.031311+0000","last_peered":"2024-09-11T05:17:06.031311+0000","last_clean":"2024-09-11T05:17:06.031311+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:06.031311+0000","last_undegraded":"2024-09-11T05:17:06.031311+0000","last_fullsized":"2024-09-11T05:17:06.031311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9270,"log_dups_size":0,"ondisk_log_size":9270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37945344,"num_objects":9264,"num_object_clones":0,"num_object_copies":18528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7668,"num_write_kb":30672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9111","reported_seq":9236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:06.026479+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:06.026479+0000","last_peered":"2024-09-11T05:17:06.026479+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:06.026479+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9111,"log_dups_size":0,"ondisk_log_size":9111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7514,"num_write_kb":30056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9204","reported_seq":9275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.978839+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:05.978839+0000","last_peered":"2024-09-11T05:17:05.978839+0000","last_clean":"2024-09-11T05:17:05.978839+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:05.978839+0000","last_undegraded":"2024-09-11T05:17:05.978839+0000","last_fullsized":"2024-09-11T05:17:05.978839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9204,"num_write_kb":36816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9329","reported_seq":9458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.188261+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:05.188261+0000","last_peered":"2024-09-11T05:17:05.188261+0000","last_clean":"2024-09-11T05:17:05.188261+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:05.188261+0000","last_undegraded":"2024-09-11T05:17:05.188261+0000","last_fullsized":"2024-09-11T05:17:05.188261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38182912,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9204","reported_seq":11352,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.206952+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:05.206952+0000","last_peered":"2024-09-11T05:17:05.206952+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:05.206952+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679104,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":22992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":999,"num_bytes_recovered":4091904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6598},{"shards":"0,2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9291","reported_seq":9423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.130563+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:05.130563+0000","last_peered":"2024-09-11T05:17:05.130563+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:05.130563+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9291,"log_dups_size":0,"ondisk_log_size":9291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38031360,"num_objects":9285,"num_object_clones":0,"num_object_copies":18570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5709},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9186","reported_seq":9310,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.176086+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:05.176086+0000","last_peered":"2024-09-11T05:17:05.176086+0000","last_clean":"2024-09-11T05:17:05.176086+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:05.176086+0000","last_undegraded":"2024-09-11T05:17:05.176086+0000","last_fullsized":"2024-09-11T05:17:05.176086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9319","reported_seq":9450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.998887+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:05.998887+0000","last_peered":"2024-09-11T05:17:05.998887+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:05.998887+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9319,"log_dups_size":0,"ondisk_log_size":9319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7387,"num_write_kb":29548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9063","reported_seq":9228,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.209837+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:05.209837+0000","last_peered":"2024-09-11T05:17:05.209837+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:05.209837+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9063,"log_dups_size":0,"ondisk_log_size":9063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37089280,"num_objects":9055,"num_object_clones":0,"num_object_copies":18110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":23264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9285","reported_seq":9401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:06.004520+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:06.004520+0000","last_peered":"2024-09-11T05:17:06.004520+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:06.004520+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7314,"num_write_kb":29256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5758}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9097","reported_seq":16182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.984791+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:05.984791+0000","last_peered":"2024-09-11T05:17:05.984791+0000","last_clean":"2024-09-11T05:17:05.984791+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:05.984791+0000","last_undegraded":"2024-09-11T05:17:05.984791+0000","last_fullsized":"2024-09-11T05:17:05.984791+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9097,"log_dups_size":0,"ondisk_log_size":9097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37236736,"num_objects":9091,"num_object_clones":0,"num_object_copies":18182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7000,"num_write_kb":28000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9328","reported_seq":9459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.192713+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:05.192713+0000","last_peered":"2024-09-11T05:17:05.192713+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:05.192713+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9328,"log_dups_size":0,"ondisk_log_size":9328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5703},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9065","reported_seq":9198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.198140+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:05.198140+0000","last_peered":"2024-09-11T05:17:05.198140+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:05.198140+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9065,"log_dups_size":0,"ondisk_log_size":9065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5931,"num_write_kb":23724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5562},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9121","reported_seq":9256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.159928+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:05.159928+0000","last_peered":"2024-09-11T05:17:05.159928+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:05.159928+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9121,"log_dups_size":0,"ondisk_log_size":9121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9083","reported_seq":9226,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:05.189156+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:05.189156+0000","last_peered":"2024-09-11T05:17:05.189156+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:05.189156+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9083,"log_dups_size":0,"ondisk_log_size":9083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37155433,"num_objects":9072,"num_object_clones":0,"num_object_copies":18144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6090,"num_write_kb":24360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602845801,"num_objects":147180,"num_object_clones":0,"num_object_copies":294360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2599,"num_objects_unfound":0,"num_objects_dirty":147180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110325,"num_write_kb":441300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4480,"num_bytes_recovered":18341888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226838016,"data_stored":1226757999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147261,"ondisk_log_size":147261,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660944,"kb_used_data":408512,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93710896,"statfs":{"total":96636764160,"available":95959957504,"internally_reserved":0,"allocated":418316288,"data_stored":418061434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512100,"kb_used_data":291604,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93859740,"statfs":{"total":96636764160,"available":96112373760,"internally_reserved":0,"allocated":298602496,"data_stored":298273985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":225767083},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794724,"kb_used_data":496532,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93577116,"statfs":{"total":96636764160,"available":95822966784,"internally_reserved":0,"allocated":508448768,"data_stored":508189837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,3,9,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509440000,"data_stored":509436521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298397696,"data_stored":298324637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419000320,"data_stored":418996841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:11.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 218 20 149649 149629 2.68003 1.65234 0.0136373 0.0291434 2024-09-11T05:17:12.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 219 20 150102 150082 2.67587 1.76953 0.0178306 0.029188 2024-09-11T05:17:13.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:2024-09-11T05:17:13.155977+0000 min lat: 0.00201616 max lat: 2.23343 avg lat: 0.0292302 2024-09-11T05:17:13.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T05:17:13.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 220 20 150573 150553 2.67207 1.83984 0.00777383 0.0292302 2024-09-11T05:17:14.031 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:14.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 221 20 151032 151012 2.66809 1.79297 0.0415806 0.0292729 2024-09-11T05:17:14.873 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:14.938 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-11T05:17:13.455338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611824252,"num_objects":149374,"num_object_clones":0,"num_object_copies":298748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2525,"num_objects_unfound":0,"num_objects_dirty":149374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112519,"num_write_kb":450070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4555,"num_bytes_recovered":18645011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149456,"ondisk_log_size":149456,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2005184,"kb_used_data":1216016,"kb_used_omap":12,"kb_used_meta":789107,"kb_avail":281110336,"statfs":{"total":289910292480,"available":287856984064,"internally_reserved":0,"allocated":1245200384,"data_stored":1244358088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":808045569},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,9,6,8,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18132992,"num_objects":4427,"num_object_clones":0,"num_object_copies":8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-149,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4427,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":610304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012956"},"pg_stats":[{"pgid":"2.f","version":"45'9453","reported_seq":9531,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.237170+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:10.237170+0000","last_peered":"2024-09-11T05:17:10.237170+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:10.237170+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9453,"log_dups_size":0,"ondisk_log_size":9453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9453,"num_write_kb":37812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5778},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9386","reported_seq":9488,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:10.987783+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:10.987783+0000","last_peered":"2024-09-11T05:17:10.987783+0000","last_clean":"2024-09-11T05:17:10.987783+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:10.987783+0000","last_undegraded":"2024-09-11T05:17:10.987783+0000","last_fullsized":"2024-09-11T05:17:10.987783+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9386,"log_dups_size":0,"ondisk_log_size":9386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38420480,"num_objects":9380,"num_object_clones":0,"num_object_copies":18760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7784,"num_write_kb":31136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9245","reported_seq":9370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.967976+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:10.967976+0000","last_peered":"2024-09-11T05:17:10.967976+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:10.967976+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9245,"log_dups_size":0,"ondisk_log_size":9245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37851136,"num_objects":9241,"num_object_clones":0,"num_object_copies":18482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7648,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5747}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9345","reported_seq":9416,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:11.105380+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:11.105380+0000","last_peered":"2024-09-11T05:17:11.105380+0000","last_clean":"2024-09-11T05:17:11.105380+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:11.105380+0000","last_undegraded":"2024-09-11T05:17:11.105380+0000","last_fullsized":"2024-09-11T05:17:11.105380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9345,"log_dups_size":0,"ondisk_log_size":9345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38277120,"num_objects":9345,"num_object_clones":0,"num_object_copies":18690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9345,"num_write_kb":37380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9472","reported_seq":9601,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:10.128562+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:10.128562+0000","last_peered":"2024-09-11T05:17:10.128562+0000","last_clean":"2024-09-11T05:17:10.128562+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:10.128562+0000","last_undegraded":"2024-09-11T05:17:10.128562+0000","last_fullsized":"2024-09-11T05:17:10.128562+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6306,"num_write_kb":25224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9341","reported_seq":11636,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.233867+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:10.233867+0000","last_peered":"2024-09-11T05:17:10.233867+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:10.233867+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240256,"num_objects":9336,"num_object_clones":0,"num_object_copies":18672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2525,"num_objects_unfound":0,"num_objects_dirty":9336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5885,"num_write_kb":23540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1073,"num_bytes_recovered":4395008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6809},{"shards":"0,2","objects":2525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9415","reported_seq":9546,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.213080+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:10.213080+0000","last_peered":"2024-09-11T05:17:10.213080+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:10.213080+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9415,"log_dups_size":0,"ondisk_log_size":9415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38539264,"num_objects":9409,"num_object_clones":0,"num_object_copies":18818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6580,"num_write_kb":26320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5833},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9322","reported_seq":9446,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:10.224023+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:10.224023+0000","last_peered":"2024-09-11T05:17:10.224023+0000","last_clean":"2024-09-11T05:17:10.224023+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:10.224023+0000","last_undegraded":"2024-09-11T05:17:10.224023+0000","last_fullsized":"2024-09-11T05:17:10.224023+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9322,"log_dups_size":0,"ondisk_log_size":9322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6625,"num_write_kb":26500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9448","reported_seq":9579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.999811+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:10.999811+0000","last_peered":"2024-09-11T05:17:10.999811+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:10.999811+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38670336,"num_objects":9441,"num_object_clones":0,"num_object_copies":18882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7516,"num_write_kb":30064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5916}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9206","reported_seq":9371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.095254+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:10.095254+0000","last_peered":"2024-09-11T05:17:10.095254+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:10.095254+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9206,"log_dups_size":0,"ondisk_log_size":9206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37675008,"num_objects":9198,"num_object_clones":0,"num_object_copies":18396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5959,"num_write_kb":23836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5756},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9423","reported_seq":9540,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.989910+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:10.989910+0000","last_peered":"2024-09-11T05:17:10.989910+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:10.989910+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9423,"log_dups_size":0,"ondisk_log_size":9423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7452,"num_write_kb":29808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5896}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9232","reported_seq":16317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:11.043219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:11.043219+0000","last_peered":"2024-09-11T05:17:11.043219+0000","last_clean":"2024-09-11T05:17:11.043219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:11.043219+0000","last_undegraded":"2024-09-11T05:17:11.043219+0000","last_fullsized":"2024-09-11T05:17:11.043219+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9232,"log_dups_size":0,"ondisk_log_size":9232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37789696,"num_objects":9226,"num_object_clones":0,"num_object_copies":18452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7135,"num_write_kb":28540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9473","reported_seq":9604,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.254623+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:10.254623+0000","last_peered":"2024-09-11T05:17:10.254623+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:10.254623+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9473,"log_dups_size":0,"ondisk_log_size":9473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38793216,"num_objects":9471,"num_object_clones":0,"num_object_copies":18942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6506,"num_write_kb":26024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5848},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9185","reported_seq":9318,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.038617+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:10.038617+0000","last_peered":"2024-09-11T05:17:10.038617+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:10.038617+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9185,"log_dups_size":0,"ondisk_log_size":9185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37588992,"num_objects":9177,"num_object_clones":0,"num_object_copies":18354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6051,"num_write_kb":24204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5682},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9277","reported_seq":9412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.194329+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:10.194329+0000","last_peered":"2024-09-11T05:17:10.194329+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:10.194329+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9277,"log_dups_size":0,"ondisk_log_size":9277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37982208,"num_objects":9273,"num_object_clones":0,"num_object_copies":18546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":24140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5729},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9230","reported_seq":9373,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.262877+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:10.262877+0000","last_peered":"2024-09-11T05:17:10.262877+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:10.262877+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9230,"log_dups_size":0,"ondisk_log_size":9230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37757545,"num_objects":9219,"num_object_clones":0,"num_object_copies":18438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5704},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611824233,"num_objects":149372,"num_object_clones":0,"num_object_copies":298744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2525,"num_objects_unfound":0,"num_objects_dirty":149372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112517,"num_write_kb":450068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4554,"num_bytes_recovered":18644992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245216768,"data_stored":1245136751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149453,"ondisk_log_size":149453,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669680,"kb_used_data":413088,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93702160,"statfs":{"total":96636764160,"available":95951011840,"internally_reserved":0,"allocated":423002112,"data_stored":422747258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524520,"kb_used_data":298456,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93847320,"statfs":{"total":96636764160,"available":96099655680,"internally_reserved":0,"allocated":305618944,"data_stored":305290433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215160,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810984,"kb_used_data":504472,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93560856,"statfs":{"total":96636764160,"available":95806316544,"internally_reserved":0,"allocated":516579328,"data_stored":516320397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,2,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516300800,"data_stored":516297321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305614848,"data_stored":305541789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423301120,"data_stored":423297641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:14.940 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:15.154 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 222 20 151468 151448 2.66375 1.70312 0.0507045 0.0293214 2024-09-11T05:17:15.838 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:15.838 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:15.954 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-11T05:17:15.462535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617652860,"num_objects":150797,"num_object_clones":0,"num_object_copies":301594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":150797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113942,"num_write_kb":455762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4623,"num_bytes_recovered":18923539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150879,"ondisk_log_size":150879,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016136,"kb_used_data":1222808,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281099384,"statfs":{"total":289910292480,"available":287845769216,"internally_reserved":0,"allocated":1252155392,"data_stored":1251313096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":812305409},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,5,9,10,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23961600,"num_objects":5850,"num_object_clones":0,"num_object_copies":11700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-217,"num_objects_unfound":0,"num_objects_dirty":5850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5850,"num_write_kb":23400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":888832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.019574"},"pg_stats":[{"pgid":"2.f","version":"45'9608","reported_seq":9686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.397653+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:15.397653+0000","last_peered":"2024-09-11T05:17:15.397653+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:15.397653+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9608,"log_dups_size":0,"ondisk_log_size":9608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9608,"num_write_kb":38432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5933},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9386","reported_seq":9488,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:10.987783+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:10.987783+0000","last_peered":"2024-09-11T05:17:10.987783+0000","last_clean":"2024-09-11T05:17:10.987783+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:10.987783+0000","last_undegraded":"2024-09-11T05:17:10.987783+0000","last_fullsized":"2024-09-11T05:17:10.987783+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9386,"log_dups_size":0,"ondisk_log_size":9386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38420480,"num_objects":9380,"num_object_clones":0,"num_object_copies":18760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7784,"num_write_kb":31136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9245","reported_seq":9370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.967976+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:10.967976+0000","last_peered":"2024-09-11T05:17:10.967976+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:10.967976+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9245,"log_dups_size":0,"ondisk_log_size":9245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37851136,"num_objects":9241,"num_object_clones":0,"num_object_copies":18482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7648,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5747}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9345","reported_seq":9416,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:11.105380+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:11.105380+0000","last_peered":"2024-09-11T05:17:11.105380+0000","last_clean":"2024-09-11T05:17:11.105380+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:11.105380+0000","last_undegraded":"2024-09-11T05:17:11.105380+0000","last_fullsized":"2024-09-11T05:17:11.105380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9345,"log_dups_size":0,"ondisk_log_size":9345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38277120,"num_objects":9345,"num_object_clones":0,"num_object_copies":18690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9345,"num_write_kb":37380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9600","reported_seq":9729,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:15.306487+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:15.306487+0000","last_peered":"2024-09-11T05:17:15.306487+0000","last_clean":"2024-09-11T05:17:15.306487+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:15.306487+0000","last_undegraded":"2024-09-11T05:17:15.306487+0000","last_fullsized":"2024-09-11T05:17:15.306487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9600,"log_dups_size":0,"ondisk_log_size":9600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9515","reported_seq":11947,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.322670+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:15.322670+0000","last_peered":"2024-09-11T05:17:15.322670+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:15.322670+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9515,"log_dups_size":0,"ondisk_log_size":9515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952960,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6059,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1141,"num_bytes_recovered":4673536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7051},{"shards":"0,2","objects":2457}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9576","reported_seq":9708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.208221+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:15.208221+0000","last_peered":"2024-09-11T05:17:15.208221+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:15.208221+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9576,"log_dups_size":0,"ondisk_log_size":9576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39198720,"num_objects":9570,"num_object_clones":0,"num_object_copies":19140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5994},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9448","reported_seq":9571,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:15.346465+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:15.346465+0000","last_peered":"2024-09-11T05:17:15.346465+0000","last_clean":"2024-09-11T05:17:15.346465+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:15.346465+0000","last_undegraded":"2024-09-11T05:17:15.346465+0000","last_fullsized":"2024-09-11T05:17:15.346465+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38690816,"num_objects":9446,"num_object_clones":0,"num_object_copies":18892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6751,"num_write_kb":27004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9448","reported_seq":9579,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.999811+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:10.999811+0000","last_peered":"2024-09-11T05:17:10.999811+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:10.999811+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38670336,"num_objects":9441,"num_object_clones":0,"num_object_copies":18882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7516,"num_write_kb":30064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":5916}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9360","reported_seq":9525,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.318113+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:15.318113+0000","last_peered":"2024-09-11T05:17:15.318113+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:15.318113+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9360,"log_dups_size":0,"ondisk_log_size":9360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38305792,"num_objects":9352,"num_object_clones":0,"num_object_copies":18704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":24452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5910},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9423","reported_seq":9540,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:10.989910+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:10.989910+0000","last_peered":"2024-09-11T05:17:10.989910+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:10.989910+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9423,"log_dups_size":0,"ondisk_log_size":9423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7452,"num_write_kb":29808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":5896}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9232","reported_seq":16317,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:11.043219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:11.043219+0000","last_peered":"2024-09-11T05:17:11.043219+0000","last_clean":"2024-09-11T05:17:11.043219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:11.043219+0000","last_undegraded":"2024-09-11T05:17:11.043219+0000","last_fullsized":"2024-09-11T05:17:11.043219+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9232,"log_dups_size":0,"ondisk_log_size":9232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37789696,"num_objects":9226,"num_object_clones":0,"num_object_copies":18452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7135,"num_write_kb":28540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9592","reported_seq":9723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.280658+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:15.280658+0000","last_peered":"2024-09-11T05:17:15.280658+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:15.280658+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9592,"log_dups_size":0,"ondisk_log_size":9592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280640,"num_objects":9590,"num_object_clones":0,"num_object_copies":19180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6625,"num_write_kb":26500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5967},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9329","reported_seq":9462,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.407758+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:15.407758+0000","last_peered":"2024-09-11T05:17:15.407758+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:15.407758+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38178816,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6195,"num_write_kb":24780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5826},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9404","reported_seq":9538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.332176+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:15.332176+0000","last_peered":"2024-09-11T05:17:15.332176+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:15.332176+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38502400,"num_objects":9400,"num_object_clones":0,"num_object_copies":18800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6162,"num_write_kb":24648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5856},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9365","reported_seq":9508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.402977+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:15.402977+0000","last_peered":"2024-09-11T05:17:15.402977+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:15.402977+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9365,"log_dups_size":0,"ondisk_log_size":9365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38310505,"num_objects":9354,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6372,"num_write_kb":25488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5839},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617652841,"num_objects":150795,"num_object_clones":0,"num_object_copies":301590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":150795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113940,"num_write_kb":455760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4622,"num_bytes_recovered":18923520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1252233216,"data_stored":1252153199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150876,"ondisk_log_size":150876,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669680,"kb_used_data":413088,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93702160,"statfs":{"total":96636764160,"available":95951011840,"internally_reserved":0,"allocated":423002112,"data_stored":422747258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524520,"kb_used_data":298456,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93847320,"statfs":{"total":96636764160,"available":96099655680,"internally_reserved":0,"allocated":305618944,"data_stored":305290433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215161,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821936,"kb_used_data":511264,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93549904,"statfs":{"total":96636764160,"available":95795101696,"internally_reserved":0,"allocated":523534336,"data_stored":523275405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523317248,"data_stored":523313769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305614848,"data_stored":305541789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423301120,"data_stored":423297641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:15.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9608", 'reported_seq': 9686, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.397653+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:15.397653+0000', 'last_peered': '2024-09-11T05:17:15.397653+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:15.397653+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 9608, 'log_dups_size': 0, 'ondisk_log_size': 9608, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39354368, 'num_objects': 9608, 'num_object_clones': 0, 'num_object_copies': 19216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9608, 'num_write_kb': 38432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5933}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9245", 'reported_seq': 9370, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:10.967976+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:10.967976+0000', 'last_peered': '2024-09-11T05:17:10.967976+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:10.967976+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 9245, 'log_dups_size': 0, 'ondisk_log_size': 9245, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37851136, 'num_objects': 9241, 'num_object_clones': 0, 'num_object_copies': 18482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9241, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7648, 'num_write_kb': 30592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 5747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9515", 'reported_seq': 11947, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.322670+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:15.322670+0000', 'last_peered': '2024-09-11T05:17:15.322670+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:15.322670+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 9515, 'log_dups_size': 0, 'ondisk_log_size': 9515, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38952960, 'num_objects': 9510, 'num_object_clones': 0, 'num_object_copies': 19020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2457, 'num_objects_unfound': 0, 'num_objects_dirty': 9510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6059, 'num_write_kb': 24236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1141, 'num_bytes_recovered': 4673536, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7051}, {'shards': '0,2', 'objects': 2457}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9576", 'reported_seq': 9708, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.208221+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:15.208221+0000', 'last_peered': '2024-09-11T05:17:15.208221+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:15.208221+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 9576, 'log_dups_size': 0, 'ondisk_log_size': 9576, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39198720, 'num_objects': 9570, 'num_object_clones': 0, 'num_object_copies': 19140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6741, 'num_write_kb': 26964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5994}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:15.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9448", 'reported_seq': 9579, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:10.999811+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:10.999811+0000', 'last_peered': '2024-09-11T05:17:10.999811+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:10.999811+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 9448, 'log_dups_size': 0, 'ondisk_log_size': 9448, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38670336, 'num_objects': 9441, 'num_object_clones': 0, 'num_object_copies': 18882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9441, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7516, 'num_write_kb': 30064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 5916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9360", 'reported_seq': 9525, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.318113+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:15.318113+0000', 'last_peered': '2024-09-11T05:17:15.318113+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:15.318113+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 9360, 'log_dups_size': 0, 'ondisk_log_size': 9360, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38305792, 'num_objects': 9352, 'num_object_clones': 0, 'num_object_copies': 18704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6113, 'num_write_kb': 24452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5910}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9423", 'reported_seq': 9540, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:10.989910+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:10.989910+0000', 'last_peered': '2024-09-11T05:17:10.989910+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:10.989910+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 9423, 'log_dups_size': 0, 'ondisk_log_size': 9423, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38576128, 'num_objects': 9418, 'num_object_clones': 0, 'num_object_copies': 18836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7452, 'num_write_kb': 29808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 5896}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9592", 'reported_seq': 9723, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.280658+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:15.280658+0000', 'last_peered': '2024-09-11T05:17:15.280658+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:15.280658+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 9592, 'log_dups_size': 0, 'ondisk_log_size': 9592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39280640, 'num_objects': 9590, 'num_object_clones': 0, 'num_object_copies': 19180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6625, 'num_write_kb': 26500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5967}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9329", 'reported_seq': 9462, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.407758+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:15.407758+0000', 'last_peered': '2024-09-11T05:17:15.407758+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:15.407758+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 9329, 'log_dups_size': 0, 'ondisk_log_size': 9329, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38178816, 'num_objects': 9321, 'num_object_clones': 0, 'num_object_copies': 18642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9321, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6195, 'num_write_kb': 24780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5826}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9404", 'reported_seq': 9538, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.332176+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:15.332176+0000', 'last_peered': '2024-09-11T05:17:15.332176+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:15.332176+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 9404, 'log_dups_size': 0, 'ondisk_log_size': 9404, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38502400, 'num_objects': 9400, 'num_object_clones': 0, 'num_object_copies': 18800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6162, 'num_write_kb': 24648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5856}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:15.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9365", 'reported_seq': 9508, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:15.402977+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:15.402977+0000', 'last_peered': '2024-09-11T05:17:15.402977+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:15.402977+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 9365, 'log_dups_size': 0, 'ondisk_log_size': 9365, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38310505, 'num_objects': 9354, 'num_object_clones': 0, 'num_object_copies': 18708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6372, 'num_write_kb': 25488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5839}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:15.958 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:16.156 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 223 20 151906 151886 2.65945 1.71094 0.0121987 0.0293681 2024-09-11T05:17:17.013 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:17.013 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/301594 objects degraded (11.748%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":314,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":150797,"data_bytes":617652860,"bytes_used":2064523264,"bytes_avail":287845769216,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":301594,"degraded_ratio":0.11747912756885084,"misplaced_objects":2457,"misplaced_total":301594,"misplaced_ratio":0.008146713794040996,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":73948,"recovering_keys_per_sec":0,"num_objects_recovered":217,"num_bytes_recovered":888832,"num_keys_recovered":0,"write_bytes_sec":1993548,"read_op_per_sec":0,"write_op_per_sec":486},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:17.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:17.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:17.156 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 224 20 152328 152308 2.65494 1.64844 0.0631515 0.029417 2024-09-11T05:17:17.928 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:17.930 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:18.013 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-11T05:17:17.463046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":621253244,"num_objects":151676,"num_object_clones":0,"num_object_copies":303352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":151676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114821,"num_write_kb":459278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4623,"num_bytes_recovered":18923539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151758,"ondisk_log_size":151758,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2040124,"kb_used_data":1234312,"kb_used_omap":12,"kb_used_meta":805747,"kb_avail":281075396,"statfs":{"total":289910292480,"available":287821205504,"internally_reserved":0,"allocated":1263935488,"data_stored":1263093192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":825084929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,1,12,13,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21925888,"num_objects":5353,"num_object_clones":0,"num_object_copies":10706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-142,"num_objects_unfound":0,"num_objects_dirty":5353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5353,"num_write_kb":21412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":581632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.019669"},"pg_stats":[{"pgid":"2.f","version":"45'9608","reported_seq":9686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.397653+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:15.397653+0000","last_peered":"2024-09-11T05:17:15.397653+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:15.397653+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9608,"log_dups_size":0,"ondisk_log_size":9608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9608,"num_write_kb":38432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5933},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9516","reported_seq":9617,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:16.088698+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:16.088698+0000","last_peered":"2024-09-11T05:17:16.088698+0000","last_clean":"2024-09-11T05:17:16.088698+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:16.088698+0000","last_undegraded":"2024-09-11T05:17:16.088698+0000","last_fullsized":"2024-09-11T05:17:16.088698+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9516,"log_dups_size":0,"ondisk_log_size":9516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952960,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7914,"num_write_kb":31656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9379","reported_seq":9504,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:16.049236+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:16.049236+0000","last_peered":"2024-09-11T05:17:16.049236+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:16.049236+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9379,"log_dups_size":0,"ondisk_log_size":9379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38400000,"num_objects":9375,"num_object_clones":0,"num_object_copies":18750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7782,"num_write_kb":31128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5881}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9507","reported_seq":9578,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:16.070483+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:16.070483+0000","last_peered":"2024-09-11T05:17:16.070483+0000","last_clean":"2024-09-11T05:17:16.070483+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:16.070483+0000","last_undegraded":"2024-09-11T05:17:16.070483+0000","last_fullsized":"2024-09-11T05:17:16.070483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9507,"log_dups_size":0,"ondisk_log_size":9507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38940672,"num_objects":9507,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9507,"num_write_kb":38028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9600","reported_seq":9729,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:15.306487+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:15.306487+0000","last_peered":"2024-09-11T05:17:15.306487+0000","last_clean":"2024-09-11T05:17:15.306487+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:15.306487+0000","last_undegraded":"2024-09-11T05:17:15.306487+0000","last_fullsized":"2024-09-11T05:17:15.306487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9600,"log_dups_size":0,"ondisk_log_size":9600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9515","reported_seq":11947,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.322670+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:15.322670+0000","last_peered":"2024-09-11T05:17:15.322670+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:15.322670+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9515,"log_dups_size":0,"ondisk_log_size":9515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952960,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6059,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1141,"num_bytes_recovered":4673536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7051},{"shards":"0,2","objects":2457}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9576","reported_seq":9708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.208221+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:15.208221+0000","last_peered":"2024-09-11T05:17:15.208221+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:15.208221+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9576,"log_dups_size":0,"ondisk_log_size":9576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39198720,"num_objects":9570,"num_object_clones":0,"num_object_copies":19140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5994},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9448","reported_seq":9571,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:15.346465+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:15.346465+0000","last_peered":"2024-09-11T05:17:15.346465+0000","last_clean":"2024-09-11T05:17:15.346465+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:15.346465+0000","last_undegraded":"2024-09-11T05:17:15.346465+0000","last_fullsized":"2024-09-11T05:17:15.346465+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38690816,"num_objects":9446,"num_object_clones":0,"num_object_copies":18892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6751,"num_write_kb":27004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9585","reported_seq":9716,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:16.108081+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:16.108081+0000","last_peered":"2024-09-11T05:17:16.108081+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:16.108081+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9585,"log_dups_size":0,"ondisk_log_size":9585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39231488,"num_objects":9578,"num_object_clones":0,"num_object_copies":19156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7653,"num_write_kb":30612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9360","reported_seq":9525,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.318113+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:15.318113+0000","last_peered":"2024-09-11T05:17:15.318113+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:15.318113+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9360,"log_dups_size":0,"ondisk_log_size":9360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38305792,"num_objects":9352,"num_object_clones":0,"num_object_copies":18704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":24452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5910},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9573","reported_seq":9690,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:16.072652+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:16.072652+0000","last_peered":"2024-09-11T05:17:16.072652+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:16.072652+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9573,"log_dups_size":0,"ondisk_log_size":9573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7602,"num_write_kb":30408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6046}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9398","reported_seq":16483,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:16.014732+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:16.014732+0000","last_peered":"2024-09-11T05:17:16.014732+0000","last_clean":"2024-09-11T05:17:16.014732+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:16.014732+0000","last_undegraded":"2024-09-11T05:17:16.014732+0000","last_fullsized":"2024-09-11T05:17:16.014732+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9398,"log_dups_size":0,"ondisk_log_size":9398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7301,"num_write_kb":29204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9592","reported_seq":9723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.280658+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:15.280658+0000","last_peered":"2024-09-11T05:17:15.280658+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:15.280658+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9592,"log_dups_size":0,"ondisk_log_size":9592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280640,"num_objects":9590,"num_object_clones":0,"num_object_copies":19180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6625,"num_write_kb":26500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5967},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9329","reported_seq":9462,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.407758+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:15.407758+0000","last_peered":"2024-09-11T05:17:15.407758+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:15.407758+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38178816,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6195,"num_write_kb":24780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5826},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9404","reported_seq":9538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.332176+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:15.332176+0000","last_peered":"2024-09-11T05:17:15.332176+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:15.332176+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38502400,"num_objects":9400,"num_object_clones":0,"num_object_copies":18800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6162,"num_write_kb":24648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5856},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9365","reported_seq":9508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:15.402977+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:15.402977+0000","last_peered":"2024-09-11T05:17:15.402977+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:15.402977+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9365,"log_dups_size":0,"ondisk_log_size":9365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38310505,"num_objects":9354,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6372,"num_write_kb":25488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5839},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":621253225,"num_objects":151674,"num_object_clones":0,"num_object_copies":303348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":151674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114819,"num_write_kb":459276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4622,"num_bytes_recovered":18923520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1264095232,"data_stored":1264015215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151755,"ondisk_log_size":151755,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678212,"kb_used_data":417460,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93693628,"statfs":{"total":96636764160,"available":95942275072,"internally_reserved":0,"allocated":427479040,"data_stored":427224186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":266989227},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539976,"kb_used_data":305588,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93831864,"statfs":{"total":96636764160,"available":96083828736,"internally_reserved":0,"allocated":312922112,"data_stored":312593601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215161,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821936,"kb_used_data":511264,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93549904,"statfs":{"total":96636764160,"available":95795101696,"internally_reserved":0,"allocated":523534336,"data_stored":523275405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523317248,"data_stored":523313769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312832000,"data_stored":312758941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":427945984,"data_stored":427942505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:18.157 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 225 20 152761 152741 2.65066 1.69141 0.0165687 0.0294653 2024-09-11T05:17:19.157 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 226 20 153206 153186 2.64662 1.73828 0.0181189 0.0295099 2024-09-11T05:17:20.157 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 227 20 153661 153641 2.64279 1.77734 0.0604164 0.0295537 2024-09-11T05:17:21.015 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:21.159 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 228 20 154122 154102 2.63908 1.80078 0.0511892 0.0295956 2024-09-11T05:17:22.004 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:22.090 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-11T05:17:21.472214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630456956,"num_objects":153923,"num_object_clones":0,"num_object_copies":307846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":153923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117068,"num_write_kb":468266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4705,"num_bytes_recovered":19259411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154005,"ondisk_log_size":154005,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049172,"kb_used_data":1246112,"kb_used_omap":12,"kb_used_meta":802995,"kb_avail":281066348,"statfs":{"total":289910292480,"available":287811940352,"internally_reserved":0,"allocated":1276018688,"data_stored":1275176392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":822266881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,2,3,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020948"},"pg_stats":[{"pgid":"2.f","version":"45'9726","reported_seq":9804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.379941+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:20.379941+0000","last_peered":"2024-09-11T05:17:20.379941+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:20.379941+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9726,"log_dups_size":0,"ondisk_log_size":9726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9726,"num_write_kb":38904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6051},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9648","reported_seq":9750,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.148861+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:21.148861+0000","last_peered":"2024-09-11T05:17:21.148861+0000","last_clean":"2024-09-11T05:17:21.148861+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:21.148861+0000","last_undegraded":"2024-09-11T05:17:21.148861+0000","last_fullsized":"2024-09-11T05:17:21.148861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9648,"log_dups_size":0,"ondisk_log_size":9648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39493632,"num_objects":9642,"num_object_clones":0,"num_object_copies":19284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9544","reported_seq":9669,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.157104+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:21.157104+0000","last_peered":"2024-09-11T05:17:21.157104+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:21.157104+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9544,"log_dups_size":0,"ondisk_log_size":9544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39075840,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7947,"num_write_kb":31788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6046}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9648","reported_seq":9719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.126776+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:21.126776+0000","last_peered":"2024-09-11T05:17:21.126776+0000","last_clean":"2024-09-11T05:17:21.126776+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:21.126776+0000","last_undegraded":"2024-09-11T05:17:21.126776+0000","last_fullsized":"2024-09-11T05:17:21.126776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9648,"log_dups_size":0,"ondisk_log_size":9648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518208,"num_objects":9648,"num_object_clones":0,"num_object_copies":19296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9648,"num_write_kb":38592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9741","reported_seq":9869,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:20.471587+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:20.471587+0000","last_peered":"2024-09-11T05:17:20.471587+0000","last_clean":"2024-09-11T05:17:20.471587+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:20.471587+0000","last_undegraded":"2024-09-11T05:17:20.471587+0000","last_fullsized":"2024-09-11T05:17:20.471587+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9741,"log_dups_size":0,"ondisk_log_size":9741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39870464,"num_objects":9734,"num_object_clones":0,"num_object_copies":19468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6575,"num_write_kb":26300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9652","reported_seq":12247,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.447520+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:20.447520+0000","last_peered":"2024-09-11T05:17:20.447520+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:20.447520+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9652,"log_dups_size":0,"ondisk_log_size":9652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39514112,"num_objects":9647,"num_object_clones":0,"num_object_copies":19294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":9647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6196,"num_write_kb":24784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1223,"num_bytes_recovered":5009408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7270},{"shards":"0,2","objects":2375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9727","reported_seq":9856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.392173+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:20.392173+0000","last_peered":"2024-09-11T05:17:20.392173+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:20.392173+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9727,"log_dups_size":0,"ondisk_log_size":9727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39817216,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6892,"num_write_kb":27568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6145},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9588","reported_seq":9712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:20.420345+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:20.420345+0000","last_peered":"2024-09-11T05:17:20.420345+0000","last_clean":"2024-09-11T05:17:20.420345+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:20.420345+0000","last_undegraded":"2024-09-11T05:17:20.420345+0000","last_fullsized":"2024-09-11T05:17:20.420345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9588,"log_dups_size":0,"ondisk_log_size":9588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39264256,"num_objects":9586,"num_object_clones":0,"num_object_copies":19172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6891,"num_write_kb":27564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9706","reported_seq":9837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.017458+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:21.017458+0000","last_peered":"2024-09-11T05:17:21.017458+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:21.017458+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39727104,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7774,"num_write_kb":31096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9507","reported_seq":9672,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.352668+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:20.352668+0000","last_peered":"2024-09-11T05:17:20.352668+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:20.352668+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9507,"log_dups_size":0,"ondisk_log_size":9507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38907904,"num_objects":9499,"num_object_clones":0,"num_object_copies":18998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6057},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9710","reported_seq":9827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.104779+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:21.104779+0000","last_peered":"2024-09-11T05:17:21.104779+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:21.104779+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9710,"log_dups_size":0,"ondisk_log_size":9710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39751680,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7739,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9551","reported_seq":16636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.060017+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:21.060017+0000","last_peered":"2024-09-11T05:17:21.060017+0000","last_clean":"2024-09-11T05:17:21.060017+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:21.060017+0000","last_undegraded":"2024-09-11T05:17:21.060017+0000","last_fullsized":"2024-09-11T05:17:21.060017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7454,"num_write_kb":29816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9732","reported_seq":9861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.391478+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:20.391478+0000","last_peered":"2024-09-11T05:17:20.391478+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:20.391478+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9732,"log_dups_size":0,"ondisk_log_size":9732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39854080,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6107},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9483","reported_seq":9617,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.449044+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:20.449044+0000","last_peered":"2024-09-11T05:17:20.449044+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:20.449044+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38809600,"num_objects":9475,"num_object_clones":0,"num_object_copies":18950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5980},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9557","reported_seq":9692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.457582+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:20.457582+0000","last_peered":"2024-09-11T05:17:20.457582+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:20.457582+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9557,"log_dups_size":0,"ondisk_log_size":9557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39129088,"num_objects":9553,"num_object_clones":0,"num_object_copies":19106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6009},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9482","reported_seq":9626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.420498+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:20.420498+0000","last_peered":"2024-09-11T05:17:20.420498+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:20.420498+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9482,"log_dups_size":0,"ondisk_log_size":9482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38789737,"num_objects":9471,"num_object_clones":0,"num_object_copies":18942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5956},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":630456937,"num_objects":153921,"num_object_clones":0,"num_object_copies":307842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":153921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117066,"num_write_kb":468264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4704,"num_bytes_recovered":19259392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1275486208,"data_stored":1275406191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154002,"ondisk_log_size":154002,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671024,"kb_used_data":422752,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93700816,"statfs":{"total":96636764160,"available":95949635584,"internally_reserved":0,"allocated":432898048,"data_stored":432643194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539976,"kb_used_data":305588,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93831864,"statfs":{"total":96636764160,"available":96083828736,"internally_reserved":0,"allocated":312922112,"data_stored":312593601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838172,"kb_used_data":517772,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93533668,"statfs":{"total":96636764160,"available":95778476032,"internally_reserved":0,"allocated":530198528,"data_stored":529939597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,0,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530083840,"data_stored":530080361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312832000,"data_stored":312758941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432570368,"data_stored":432566889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:22.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:22.162 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 229 20 154547 154527 2.63476 1.66016 0.0141744 0.0296426 2024-09-11T05:17:22.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:23.000 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:23.048 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-11T05:17:21.472214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630456956,"num_objects":153923,"num_object_clones":0,"num_object_copies":307846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":153923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117068,"num_write_kb":468266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4705,"num_bytes_recovered":19259411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154005,"ondisk_log_size":154005,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2049172,"kb_used_data":1246112,"kb_used_omap":12,"kb_used_meta":802995,"kb_avail":281066348,"statfs":{"total":289910292480,"available":287811940352,"internally_reserved":0,"allocated":1276018688,"data_stored":1275176392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":822266881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,2,3,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-224,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020948"},"pg_stats":[{"pgid":"2.f","version":"45'9726","reported_seq":9804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.379941+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:20.379941+0000","last_peered":"2024-09-11T05:17:20.379941+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:20.379941+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9726,"log_dups_size":0,"ondisk_log_size":9726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9726,"num_write_kb":38904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6051},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9648","reported_seq":9750,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.148861+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:21.148861+0000","last_peered":"2024-09-11T05:17:21.148861+0000","last_clean":"2024-09-11T05:17:21.148861+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:21.148861+0000","last_undegraded":"2024-09-11T05:17:21.148861+0000","last_fullsized":"2024-09-11T05:17:21.148861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9648,"log_dups_size":0,"ondisk_log_size":9648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39493632,"num_objects":9642,"num_object_clones":0,"num_object_copies":19284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9544","reported_seq":9669,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.157104+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:21.157104+0000","last_peered":"2024-09-11T05:17:21.157104+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:21.157104+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9544,"log_dups_size":0,"ondisk_log_size":9544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39075840,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7947,"num_write_kb":31788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6046}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9648","reported_seq":9719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.126776+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:21.126776+0000","last_peered":"2024-09-11T05:17:21.126776+0000","last_clean":"2024-09-11T05:17:21.126776+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:21.126776+0000","last_undegraded":"2024-09-11T05:17:21.126776+0000","last_fullsized":"2024-09-11T05:17:21.126776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9648,"log_dups_size":0,"ondisk_log_size":9648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518208,"num_objects":9648,"num_object_clones":0,"num_object_copies":19296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9648,"num_write_kb":38592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9741","reported_seq":9869,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:20.471587+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:20.471587+0000","last_peered":"2024-09-11T05:17:20.471587+0000","last_clean":"2024-09-11T05:17:20.471587+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:20.471587+0000","last_undegraded":"2024-09-11T05:17:20.471587+0000","last_fullsized":"2024-09-11T05:17:20.471587+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9741,"log_dups_size":0,"ondisk_log_size":9741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39870464,"num_objects":9734,"num_object_clones":0,"num_object_copies":19468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6575,"num_write_kb":26300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9652","reported_seq":12247,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.447520+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:20.447520+0000","last_peered":"2024-09-11T05:17:20.447520+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:20.447520+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9652,"log_dups_size":0,"ondisk_log_size":9652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39514112,"num_objects":9647,"num_object_clones":0,"num_object_copies":19294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":9647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6196,"num_write_kb":24784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1223,"num_bytes_recovered":5009408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7270},{"shards":"0,2","objects":2375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9727","reported_seq":9856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.392173+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:20.392173+0000","last_peered":"2024-09-11T05:17:20.392173+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:20.392173+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9727,"log_dups_size":0,"ondisk_log_size":9727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39817216,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6892,"num_write_kb":27568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6145},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9588","reported_seq":9712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:20.420345+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:20.420345+0000","last_peered":"2024-09-11T05:17:20.420345+0000","last_clean":"2024-09-11T05:17:20.420345+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:20.420345+0000","last_undegraded":"2024-09-11T05:17:20.420345+0000","last_fullsized":"2024-09-11T05:17:20.420345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9588,"log_dups_size":0,"ondisk_log_size":9588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39264256,"num_objects":9586,"num_object_clones":0,"num_object_copies":19172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6891,"num_write_kb":27564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9706","reported_seq":9837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.017458+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:21.017458+0000","last_peered":"2024-09-11T05:17:21.017458+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:21.017458+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39727104,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7774,"num_write_kb":31096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9507","reported_seq":9672,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.352668+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:20.352668+0000","last_peered":"2024-09-11T05:17:20.352668+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:20.352668+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9507,"log_dups_size":0,"ondisk_log_size":9507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38907904,"num_objects":9499,"num_object_clones":0,"num_object_copies":18998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6057},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9710","reported_seq":9827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.104779+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:21.104779+0000","last_peered":"2024-09-11T05:17:21.104779+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:21.104779+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9710,"log_dups_size":0,"ondisk_log_size":9710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39751680,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7739,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9551","reported_seq":16636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.060017+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:21.060017+0000","last_peered":"2024-09-11T05:17:21.060017+0000","last_clean":"2024-09-11T05:17:21.060017+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:21.060017+0000","last_undegraded":"2024-09-11T05:17:21.060017+0000","last_fullsized":"2024-09-11T05:17:21.060017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7454,"num_write_kb":29816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9732","reported_seq":9861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.391478+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:20.391478+0000","last_peered":"2024-09-11T05:17:20.391478+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:20.391478+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9732,"log_dups_size":0,"ondisk_log_size":9732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39854080,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6107},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9483","reported_seq":9617,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.449044+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:20.449044+0000","last_peered":"2024-09-11T05:17:20.449044+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:20.449044+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38809600,"num_objects":9475,"num_object_clones":0,"num_object_copies":18950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5980},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9557","reported_seq":9692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.457582+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:20.457582+0000","last_peered":"2024-09-11T05:17:20.457582+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:20.457582+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9557,"log_dups_size":0,"ondisk_log_size":9557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39129088,"num_objects":9553,"num_object_clones":0,"num_object_copies":19106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6009},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9482","reported_seq":9626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.420498+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:20.420498+0000","last_peered":"2024-09-11T05:17:20.420498+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:20.420498+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9482,"log_dups_size":0,"ondisk_log_size":9482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38789737,"num_objects":9471,"num_object_clones":0,"num_object_copies":18942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5956},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":630456937,"num_objects":153921,"num_object_clones":0,"num_object_copies":307842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":153921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117066,"num_write_kb":468264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4704,"num_bytes_recovered":19259392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1275486208,"data_stored":1275406191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154002,"ondisk_log_size":154002,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671024,"kb_used_data":422752,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93700816,"statfs":{"total":96636764160,"available":95949635584,"internally_reserved":0,"allocated":432898048,"data_stored":432643194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539976,"kb_used_data":305588,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93831864,"statfs":{"total":96636764160,"available":96083828736,"internally_reserved":0,"allocated":312922112,"data_stored":312593601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838172,"kb_used_data":517772,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93533668,"statfs":{"total":96636764160,"available":95778476032,"internally_reserved":0,"allocated":530198528,"data_stored":529939597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,0,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530083840,"data_stored":530080361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312832000,"data_stored":312758941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432570368,"data_stored":432566889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:23.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:23.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9726", 'reported_seq': 9804, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.379941+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:20.379941+0000', 'last_peered': '2024-09-11T05:17:20.379941+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:20.379941+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 9726, 'log_dups_size': 0, 'ondisk_log_size': 9726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39837696, 'num_objects': 9726, 'num_object_clones': 0, 'num_object_copies': 19452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9726, 'num_write_kb': 38904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6051}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:23.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9544", 'reported_seq': 9669, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:21.157104+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:21.157104+0000', 'last_peered': '2024-09-11T05:17:21.157104+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:21.157104+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 9544, 'log_dups_size': 0, 'ondisk_log_size': 9544, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39075840, 'num_objects': 9540, 'num_object_clones': 0, 'num_object_copies': 19080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9540, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7947, 'num_write_kb': 31788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 6046}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9652", 'reported_seq': 12247, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.447520+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:20.447520+0000', 'last_peered': '2024-09-11T05:17:20.447520+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:20.447520+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 9652, 'log_dups_size': 0, 'ondisk_log_size': 9652, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39514112, 'num_objects': 9647, 'num_object_clones': 0, 'num_object_copies': 19294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2375, 'num_objects_unfound': 0, 'num_objects_dirty': 9647, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6196, 'num_write_kb': 24784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1223, 'num_bytes_recovered': 5009408, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7270}, {'shards': '0,2', 'objects': 2375}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9727", 'reported_seq': 9856, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.392173+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:20.392173+0000', 'last_peered': '2024-09-11T05:17:20.392173+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:20.392173+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 9727, 'log_dups_size': 0, 'ondisk_log_size': 9727, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39817216, 'num_objects': 9721, 'num_object_clones': 0, 'num_object_copies': 19442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6892, 'num_write_kb': 27568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6145}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9706", 'reported_seq': 9837, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:21.017458+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:21.017458+0000', 'last_peered': '2024-09-11T05:17:21.017458+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:21.017458+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 9706, 'log_dups_size': 0, 'ondisk_log_size': 9706, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39727104, 'num_objects': 9699, 'num_object_clones': 0, 'num_object_copies': 19398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9699, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7774, 'num_write_kb': 31096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 6174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9507", 'reported_seq': 9672, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.352668+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:20.352668+0000', 'last_peered': '2024-09-11T05:17:20.352668+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:20.352668+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 9507, 'log_dups_size': 0, 'ondisk_log_size': 9507, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38907904, 'num_objects': 9499, 'num_object_clones': 0, 'num_object_copies': 18998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9499, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6260, 'num_write_kb': 25040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6057}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9710", 'reported_seq': 9827, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:21.104779+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:21.104779+0000', 'last_peered': '2024-09-11T05:17:21.104779+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:21.104779+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 9710, 'log_dups_size': 0, 'ondisk_log_size': 9710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39751680, 'num_objects': 9705, 'num_object_clones': 0, 'num_object_copies': 19410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7739, 'num_write_kb': 30956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 6183}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9732", 'reported_seq': 9861, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.391478+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:20.391478+0000', 'last_peered': '2024-09-11T05:17:20.391478+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:20.391478+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 9732, 'log_dups_size': 0, 'ondisk_log_size': 9732, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39854080, 'num_objects': 9730, 'num_object_clones': 0, 'num_object_copies': 19460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6765, 'num_write_kb': 27060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6107}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9483", 'reported_seq': 9617, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.449044+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:20.449044+0000', 'last_peered': '2024-09-11T05:17:20.449044+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:20.449044+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 9483, 'log_dups_size': 0, 'ondisk_log_size': 9483, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38809600, 'num_objects': 9475, 'num_object_clones': 0, 'num_object_copies': 18950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9475, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6349, 'num_write_kb': 25396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5980}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9557", 'reported_seq': 9692, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.457582+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:20.457582+0000', 'last_peered': '2024-09-11T05:17:20.457582+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:20.457582+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 9557, 'log_dups_size': 0, 'ondisk_log_size': 9557, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39129088, 'num_objects': 9553, 'num_object_clones': 0, 'num_object_copies': 19106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9553, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6315, 'num_write_kb': 25260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6009}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:23.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9482", 'reported_seq': 9626, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:20.420498+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:20.420498+0000', 'last_peered': '2024-09-11T05:17:20.420498+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:20.420498+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 9482, 'log_dups_size': 0, 'ondisk_log_size': 9482, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38789737, 'num_objects': 9471, 'num_object_clones': 0, 'num_object_copies': 18942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6489, 'num_write_kb': 25956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5956}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:23.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:23.162 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 230 20 154981 154961 2.63068 1.69531 0.0230395 0.0296905 2024-09-11T05:17:24.162 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 231 20 155418 155398 2.62668 1.70703 0.0555195 0.0297362 2024-09-11T05:17:24.241 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:24.241 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/307846 objects degraded (11.509%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":322,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153923,"data_bytes":630456956,"bytes_used":2109730816,"bytes_avail":287800561664,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":307846,"degraded_ratio":0.11509326091617238,"misplaced_objects":2375,"misplaced_total":307846,"misplaced_ratio":0.0077148964092435832,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":51116,"recovering_keys_per_sec":0,"num_objects_recovered":150,"num_bytes_recovered":614400,"num_keys_recovered":0,"write_bytes_sec":1550203,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:24.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:24.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:25.135 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:25.162 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 232 20 155833 155813 2.62235 1.62109 0.0118844 0.0297839 2024-09-11T05:17:25.208 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-11T05:17:23.474418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630456956,"num_objects":153923,"num_object_clones":0,"num_object_copies":307846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":153923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117068,"num_write_kb":468266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4705,"num_bytes_recovered":19259411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154005,"ondisk_log_size":154005,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2060284,"kb_used_data":1253068,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281055236,"statfs":{"total":289910292480,"available":287800561664,"internally_reserved":0,"allocated":1283141632,"data_stored":1282299336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":826526721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,2,0,9,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18632704,"num_objects":4549,"num_object_clones":0,"num_object_copies":9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-150,"num_objects_unfound":0,"num_objects_dirty":4549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4549,"num_write_kb":18196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":614400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.019523"},"pg_stats":[{"pgid":"2.f","version":"45'9726","reported_seq":9804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.379941+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:20.379941+0000","last_peered":"2024-09-11T05:17:20.379941+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:20.379941+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9726,"log_dups_size":0,"ondisk_log_size":9726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9726,"num_write_kb":38904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6051},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9648","reported_seq":9750,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.148861+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:21.148861+0000","last_peered":"2024-09-11T05:17:21.148861+0000","last_clean":"2024-09-11T05:17:21.148861+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:21.148861+0000","last_undegraded":"2024-09-11T05:17:21.148861+0000","last_fullsized":"2024-09-11T05:17:21.148861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9648,"log_dups_size":0,"ondisk_log_size":9648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39493632,"num_objects":9642,"num_object_clones":0,"num_object_copies":19284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9544","reported_seq":9669,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.157104+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:21.157104+0000","last_peered":"2024-09-11T05:17:21.157104+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:21.157104+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9544,"log_dups_size":0,"ondisk_log_size":9544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39075840,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7947,"num_write_kb":31788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6046}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9648","reported_seq":9719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.126776+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:21.126776+0000","last_peered":"2024-09-11T05:17:21.126776+0000","last_clean":"2024-09-11T05:17:21.126776+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:21.126776+0000","last_undegraded":"2024-09-11T05:17:21.126776+0000","last_fullsized":"2024-09-11T05:17:21.126776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9648,"log_dups_size":0,"ondisk_log_size":9648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518208,"num_objects":9648,"num_object_clones":0,"num_object_copies":19296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9648,"num_write_kb":38592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9741","reported_seq":9869,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:20.471587+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:20.471587+0000","last_peered":"2024-09-11T05:17:20.471587+0000","last_clean":"2024-09-11T05:17:20.471587+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:20.471587+0000","last_undegraded":"2024-09-11T05:17:20.471587+0000","last_fullsized":"2024-09-11T05:17:20.471587+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9741,"log_dups_size":0,"ondisk_log_size":9741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39870464,"num_objects":9734,"num_object_clones":0,"num_object_copies":19468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6575,"num_write_kb":26300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9652","reported_seq":12247,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.447520+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:20.447520+0000","last_peered":"2024-09-11T05:17:20.447520+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:20.447520+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9652,"log_dups_size":0,"ondisk_log_size":9652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39514112,"num_objects":9647,"num_object_clones":0,"num_object_copies":19294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":9647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6196,"num_write_kb":24784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1223,"num_bytes_recovered":5009408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7270},{"shards":"0,2","objects":2375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9727","reported_seq":9856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.392173+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:20.392173+0000","last_peered":"2024-09-11T05:17:20.392173+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:20.392173+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9727,"log_dups_size":0,"ondisk_log_size":9727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39817216,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6892,"num_write_kb":27568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6145},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9588","reported_seq":9712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:20.420345+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:20.420345+0000","last_peered":"2024-09-11T05:17:20.420345+0000","last_clean":"2024-09-11T05:17:20.420345+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:20.420345+0000","last_undegraded":"2024-09-11T05:17:20.420345+0000","last_fullsized":"2024-09-11T05:17:20.420345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9588,"log_dups_size":0,"ondisk_log_size":9588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39264256,"num_objects":9586,"num_object_clones":0,"num_object_copies":19172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6891,"num_write_kb":27564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9706","reported_seq":9837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.017458+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:21.017458+0000","last_peered":"2024-09-11T05:17:21.017458+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:21.017458+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39727104,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7774,"num_write_kb":31096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566643+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566643+0000","last_peered":"2024-09-11T05:15:57.566643+0000","last_clean":"2024-09-11T05:15:57.566643+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566643+0000","last_undegraded":"2024-09-11T05:15:57.566643+0000","last_fullsized":"2024-09-11T05:15:57.566643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9507","reported_seq":9672,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.352668+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:20.352668+0000","last_peered":"2024-09-11T05:17:20.352668+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:20.352668+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9507,"log_dups_size":0,"ondisk_log_size":9507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38907904,"num_objects":9499,"num_object_clones":0,"num_object_copies":18998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6057},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9710","reported_seq":9827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:21.104779+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:21.104779+0000","last_peered":"2024-09-11T05:17:21.104779+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:21.104779+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9710,"log_dups_size":0,"ondisk_log_size":9710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39751680,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7739,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9551","reported_seq":16636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:21.060017+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:21.060017+0000","last_peered":"2024-09-11T05:17:21.060017+0000","last_clean":"2024-09-11T05:17:21.060017+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:21.060017+0000","last_undegraded":"2024-09-11T05:17:21.060017+0000","last_fullsized":"2024-09-11T05:17:21.060017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7454,"num_write_kb":29816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9732","reported_seq":9861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.391478+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:20.391478+0000","last_peered":"2024-09-11T05:17:20.391478+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:20.391478+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9732,"log_dups_size":0,"ondisk_log_size":9732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39854080,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6107},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9483","reported_seq":9617,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.449044+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:20.449044+0000","last_peered":"2024-09-11T05:17:20.449044+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:20.449044+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38809600,"num_objects":9475,"num_object_clones":0,"num_object_copies":18950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5980},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567164+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567164+0000","last_peered":"2024-09-11T05:15:57.567164+0000","last_clean":"2024-09-11T05:15:57.567164+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567164+0000","last_undegraded":"2024-09-11T05:15:57.567164+0000","last_fullsized":"2024-09-11T05:15:57.567164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9557","reported_seq":9692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.457582+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:20.457582+0000","last_peered":"2024-09-11T05:17:20.457582+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:20.457582+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9557,"log_dups_size":0,"ondisk_log_size":9557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39129088,"num_objects":9553,"num_object_clones":0,"num_object_copies":19106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6009},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9482","reported_seq":9626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:20.420498+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:20.420498+0000","last_peered":"2024-09-11T05:17:20.420498+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:20.420498+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9482,"log_dups_size":0,"ondisk_log_size":9482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38789737,"num_objects":9471,"num_object_clones":0,"num_object_copies":18942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5956},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":630456937,"num_objects":153921,"num_object_clones":0,"num_object_copies":307842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2375,"num_objects_unfound":0,"num_objects_dirty":153921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117066,"num_write_kb":468264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4704,"num_bytes_recovered":19259392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1282736128,"data_stored":1282656111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154002,"ondisk_log_size":154002,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671024,"kb_used_data":422752,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93700816,"statfs":{"total":96636764160,"available":95949635584,"internally_reserved":0,"allocated":432898048,"data_stored":432643194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551088,"kb_used_data":312544,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93820752,"statfs":{"total":96636764160,"available":96072450048,"internally_reserved":0,"allocated":320045056,"data_stored":319716545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838172,"kb_used_data":517772,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93533668,"statfs":{"total":96636764160,"available":95778476032,"internally_reserved":0,"allocated":530198528,"data_stored":529939597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,0,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530083840,"data_stored":530080361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320081920,"data_stored":320008861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432570368,"data_stored":432566889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:26.163 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 233 20 156278 156258 2.61856 1.73828 0.022193 0.0298275 2024-09-11T05:17:27.162 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 234 20 156735 156715 2.61499 1.78516 0.0851839 0.029869 2024-09-11T05:17:28.162 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 235 19 157187 157168 2.6114 1.76953 0.0637038 0.0299098 2024-09-11T05:17:28.213 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:28.966 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:28.966 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:29.021 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-11T05:17:27.475235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639308412,"num_objects":156084,"num_object_clones":0,"num_object_copies":312168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2312,"num_objects_unfound":0,"num_objects_dirty":156084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119229,"num_write_kb":476910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4768,"num_bytes_recovered":19517459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156166,"ondisk_log_size":156166,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2078612,"kb_used_data":1263076,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281036908,"statfs":{"total":289910292480,"available":287781793792,"internally_reserved":0,"allocated":1293389824,"data_stored":1292547528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":835046401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,2,6,5,8,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-145,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":593920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012700"},"pg_stats":[{"pgid":"2.f","version":"45'9866","reported_seq":9944,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.422022+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:25.422022+0000","last_peered":"2024-09-11T05:17:25.422022+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:25.422022+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9866,"log_dups_size":0,"ondisk_log_size":9866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40411136,"num_objects":9866,"num_object_clones":0,"num_object_copies":19732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9866,"num_write_kb":39464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6191},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9782","reported_seq":9884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:26.156587+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:26.156587+0000","last_peered":"2024-09-11T05:17:26.156587+0000","last_clean":"2024-09-11T05:17:26.156587+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:26.156587+0000","last_undegraded":"2024-09-11T05:17:26.156587+0000","last_fullsized":"2024-09-11T05:17:26.156587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9782,"log_dups_size":0,"ondisk_log_size":9782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40042496,"num_objects":9776,"num_object_clones":0,"num_object_copies":19552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8180,"num_write_kb":32720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9677","reported_seq":9802,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:26.115997+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:26.115997+0000","last_peered":"2024-09-11T05:17:26.115997+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:26.115997+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":32320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9790","reported_seq":9860,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:26.189923+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:26.189923+0000","last_peered":"2024-09-11T05:17:26.189923+0000","last_clean":"2024-09-11T05:17:26.189923+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:26.189923+0000","last_undegraded":"2024-09-11T05:17:26.189923+0000","last_fullsized":"2024-09-11T05:17:26.189923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9790,"log_dups_size":0,"ondisk_log_size":9790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40099840,"num_objects":9790,"num_object_clones":0,"num_object_copies":19580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9790,"num_write_kb":39160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9862","reported_seq":9992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:25.392888+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:25.392888+0000","last_peered":"2024-09-11T05:17:25.392888+0000","last_clean":"2024-09-11T05:17:25.392888+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:25.392888+0000","last_undegraded":"2024-09-11T05:17:25.392888+0000","last_fullsized":"2024-09-11T05:17:25.392888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9862,"log_dups_size":0,"ondisk_log_size":9862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40366080,"num_objects":9855,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6696,"num_write_kb":26784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9818","reported_seq":12540,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.465701+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:25.465701+0000","last_peered":"2024-09-11T05:17:25.465701+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:25.465701+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9818,"log_dups_size":0,"ondisk_log_size":9818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40194048,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2312,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6362,"num_write_kb":25448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1286,"num_bytes_recovered":5267456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7499},{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9859","reported_seq":9992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.493240+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:25.493240+0000","last_peered":"2024-09-11T05:17:25.493240+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:25.493240+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9859,"log_dups_size":0,"ondisk_log_size":9859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40357888,"num_objects":9853,"num_object_clones":0,"num_object_copies":19706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7024,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6277},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9708","reported_seq":9832,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:25.496323+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:25.496323+0000","last_peered":"2024-09-11T05:17:25.496323+0000","last_clean":"2024-09-11T05:17:25.496323+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:25.496323+0000","last_undegraded":"2024-09-11T05:17:25.496323+0000","last_fullsized":"2024-09-11T05:17:25.496323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9708,"log_dups_size":0,"ondisk_log_size":9708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7011,"num_write_kb":28044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9835","reported_seq":9966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:26.044107+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:26.044107+0000","last_peered":"2024-09-11T05:17:26.044107+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:26.044107+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9649","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.468571+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:25.468571+0000","last_peered":"2024-09-11T05:17:25.468571+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:25.468571+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9649,"log_dups_size":0,"ondisk_log_size":9649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39489536,"num_objects":9641,"num_object_clones":0,"num_object_copies":19282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6402,"num_write_kb":25608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6199},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9837","reported_seq":9955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:26.202050+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:26.202050+0000","last_peered":"2024-09-11T05:17:26.202050+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:26.202050+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9837,"log_dups_size":0,"ondisk_log_size":9837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9683","reported_seq":16768,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:26.151855+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:26.151855+0000","last_peered":"2024-09-11T05:17:26.151855+0000","last_clean":"2024-09-11T05:17:26.151855+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:26.151855+0000","last_undegraded":"2024-09-11T05:17:26.151855+0000","last_fullsized":"2024-09-11T05:17:26.151855+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7586,"num_write_kb":30344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9866","reported_seq":9997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.505785+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:25.505785+0000","last_peered":"2024-09-11T05:17:25.505785+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:25.505785+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9866,"log_dups_size":0,"ondisk_log_size":9866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6899,"num_write_kb":27596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6241},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9628","reported_seq":9761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.480348+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:25.480348+0000","last_peered":"2024-09-11T05:17:25.480348+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:25.480348+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9628,"log_dups_size":0,"ondisk_log_size":9628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6125},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9696","reported_seq":9828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.502309+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:25.502309+0000","last_peered":"2024-09-11T05:17:25.502309+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:25.502309+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39698432,"num_objects":9692,"num_object_clones":0,"num_object_copies":19384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6148},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9607","reported_seq":9751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.432525+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:25.432525+0000","last_peered":"2024-09-11T05:17:25.432525+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:25.432525+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9607,"log_dups_size":0,"ondisk_log_size":9607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301737,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6614,"num_write_kb":26456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6081},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":639308393,"num_objects":156082,"num_object_clones":0,"num_object_copies":312164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2312,"num_objects_unfound":0,"num_objects_dirty":156082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119227,"num_write_kb":476908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4767,"num_bytes_recovered":19517440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1293750272,"data_stored":1293670255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156163,"ondisk_log_size":156163,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678652,"kb_used_data":426220,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93693188,"statfs":{"total":96636764160,"available":95941824512,"internally_reserved":0,"allocated":436449280,"data_stored":436194426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551088,"kb_used_data":312544,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93820752,"statfs":{"total":96636764160,"available":96072450048,"internally_reserved":0,"allocated":320045056,"data_stored":319716545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215163,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848872,"kb_used_data":524312,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93522968,"statfs":{"total":96636764160,"available":95767519232,"internally_reserved":0,"allocated":536895488,"data_stored":536636557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332328619},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536809472,"data_stored":536805993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320081920,"data_stored":320008861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":436858880,"data_stored":436855401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:29.023 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:29.162 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 236 20 157597 157577 2.6071 1.59766 0.141369 0.0299538 2024-09-11T05:17:29.768 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:29.769 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:29.823 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-11T05:17:29.475768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639308412,"num_objects":156084,"num_object_clones":0,"num_object_copies":312168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2312,"num_objects_unfound":0,"num_objects_dirty":156084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119229,"num_write_kb":476910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4768,"num_bytes_recovered":19517459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156166,"ondisk_log_size":156166,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2073004,"kb_used_data":1269948,"kb_used_omap":12,"kb_used_meta":802995,"kb_avail":281042516,"statfs":{"total":289910292480,"available":287787536384,"internally_reserved":0,"allocated":1300426752,"data_stored":1299584456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":822266881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,7,7,4,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18055168,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-145,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4408,"num_write_kb":17632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":593920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012722"},"pg_stats":[{"pgid":"2.f","version":"45'9866","reported_seq":9944,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.422022+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:25.422022+0000","last_peered":"2024-09-11T05:17:25.422022+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:25.422022+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9866,"log_dups_size":0,"ondisk_log_size":9866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40411136,"num_objects":9866,"num_object_clones":0,"num_object_copies":19732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9866,"num_write_kb":39464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6191},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9782","reported_seq":9884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:26.156587+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:26.156587+0000","last_peered":"2024-09-11T05:17:26.156587+0000","last_clean":"2024-09-11T05:17:26.156587+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:26.156587+0000","last_undegraded":"2024-09-11T05:17:26.156587+0000","last_fullsized":"2024-09-11T05:17:26.156587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9782,"log_dups_size":0,"ondisk_log_size":9782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40042496,"num_objects":9776,"num_object_clones":0,"num_object_copies":19552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8180,"num_write_kb":32720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9677","reported_seq":9802,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:26.115997+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:26.115997+0000","last_peered":"2024-09-11T05:17:26.115997+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:26.115997+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":32320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9790","reported_seq":9860,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:26.189923+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:26.189923+0000","last_peered":"2024-09-11T05:17:26.189923+0000","last_clean":"2024-09-11T05:17:26.189923+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:26.189923+0000","last_undegraded":"2024-09-11T05:17:26.189923+0000","last_fullsized":"2024-09-11T05:17:26.189923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9790,"log_dups_size":0,"ondisk_log_size":9790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40099840,"num_objects":9790,"num_object_clones":0,"num_object_copies":19580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9790,"num_write_kb":39160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9862","reported_seq":9992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:25.392888+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:25.392888+0000","last_peered":"2024-09-11T05:17:25.392888+0000","last_clean":"2024-09-11T05:17:25.392888+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:25.392888+0000","last_undegraded":"2024-09-11T05:17:25.392888+0000","last_fullsized":"2024-09-11T05:17:25.392888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":9862,"log_dups_size":0,"ondisk_log_size":9862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40366080,"num_objects":9855,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6696,"num_write_kb":26784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9818","reported_seq":12540,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.465701+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:25.465701+0000","last_peered":"2024-09-11T05:17:25.465701+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:25.465701+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9818,"log_dups_size":0,"ondisk_log_size":9818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40194048,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2312,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6362,"num_write_kb":25448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1286,"num_bytes_recovered":5267456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7499},{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9859","reported_seq":9992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.493240+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:25.493240+0000","last_peered":"2024-09-11T05:17:25.493240+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:25.493240+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":9859,"log_dups_size":0,"ondisk_log_size":9859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40357888,"num_objects":9853,"num_object_clones":0,"num_object_copies":19706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7024,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6277},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9708","reported_seq":9832,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:25.496323+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:25.496323+0000","last_peered":"2024-09-11T05:17:25.496323+0000","last_clean":"2024-09-11T05:17:25.496323+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:25.496323+0000","last_undegraded":"2024-09-11T05:17:25.496323+0000","last_fullsized":"2024-09-11T05:17:25.496323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9708,"log_dups_size":0,"ondisk_log_size":9708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7011,"num_write_kb":28044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9835","reported_seq":9966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:26.044107+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:26.044107+0000","last_peered":"2024-09-11T05:17:26.044107+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:26.044107+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9649","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.468571+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:25.468571+0000","last_peered":"2024-09-11T05:17:25.468571+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:25.468571+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9649,"log_dups_size":0,"ondisk_log_size":9649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39489536,"num_objects":9641,"num_object_clones":0,"num_object_copies":19282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6402,"num_write_kb":25608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6199},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9837","reported_seq":9955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:26.202050+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:26.202050+0000","last_peered":"2024-09-11T05:17:26.202050+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:26.202050+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9837,"log_dups_size":0,"ondisk_log_size":9837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9683","reported_seq":16768,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:26.151855+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:26.151855+0000","last_peered":"2024-09-11T05:17:26.151855+0000","last_clean":"2024-09-11T05:17:26.151855+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:26.151855+0000","last_undegraded":"2024-09-11T05:17:26.151855+0000","last_fullsized":"2024-09-11T05:17:26.151855+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7586,"num_write_kb":30344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9866","reported_seq":9997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.505785+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:25.505785+0000","last_peered":"2024-09-11T05:17:25.505785+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:25.505785+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":9866,"log_dups_size":0,"ondisk_log_size":9866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6899,"num_write_kb":27596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6241},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9628","reported_seq":9761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.480348+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:25.480348+0000","last_peered":"2024-09-11T05:17:25.480348+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:25.480348+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9628,"log_dups_size":0,"ondisk_log_size":9628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6494,"num_write_kb":25976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6125},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9696","reported_seq":9828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.502309+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:25.502309+0000","last_peered":"2024-09-11T05:17:25.502309+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:25.502309+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39698432,"num_objects":9692,"num_object_clones":0,"num_object_copies":19384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6148},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9607","reported_seq":9751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:25.432525+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:25.432525+0000","last_peered":"2024-09-11T05:17:25.432525+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:25.432525+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9607,"log_dups_size":0,"ondisk_log_size":9607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301737,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6614,"num_write_kb":26456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6081},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":639308393,"num_objects":156082,"num_object_clones":0,"num_object_copies":312164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2312,"num_objects_unfound":0,"num_objects_dirty":156082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119227,"num_write_kb":476908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4767,"num_bytes_recovered":19517440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300774912,"data_stored":1300694895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156163,"ondisk_log_size":156163,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678652,"kb_used_data":426220,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93693188,"statfs":{"total":96636764160,"available":95941824512,"internally_reserved":0,"allocated":436449280,"data_stored":436194426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545480,"kb_used_data":319416,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93826360,"statfs":{"total":96636764160,"available":96078192640,"internally_reserved":0,"allocated":327081984,"data_stored":326753473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215163,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848872,"kb_used_data":524312,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93522968,"statfs":{"total":96636764160,"available":95767519232,"internally_reserved":0,"allocated":536895488,"data_stored":536636557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332328619},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536809472,"data_stored":536805993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327106560,"data_stored":327033501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":436858880,"data_stored":436855401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:29.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:29.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9866", 'reported_seq': 9944, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.422022+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:25.422022+0000', 'last_peered': '2024-09-11T05:17:25.422022+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:25.422022+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 9866, 'log_dups_size': 0, 'ondisk_log_size': 9866, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40411136, 'num_objects': 9866, 'num_object_clones': 0, 'num_object_copies': 19732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9866, 'num_write_kb': 39464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6191}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:29.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9677", 'reported_seq': 9802, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:26.115997+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:26.115997+0000', 'last_peered': '2024-09-11T05:17:26.115997+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:26.115997+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 9677, 'log_dups_size': 0, 'ondisk_log_size': 9677, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39620608, 'num_objects': 9673, 'num_object_clones': 0, 'num_object_copies': 19346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9673, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8080, 'num_write_kb': 32320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 6179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:29.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:29.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9818", 'reported_seq': 12540, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.465701+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:25.465701+0000', 'last_peered': '2024-09-11T05:17:25.465701+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:25.465701+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 9818, 'log_dups_size': 0, 'ondisk_log_size': 9818, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40194048, 'num_objects': 9813, 'num_object_clones': 0, 'num_object_copies': 19626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2312, 'num_objects_unfound': 0, 'num_objects_dirty': 9813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6362, 'num_write_kb': 25448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1286, 'num_bytes_recovered': 5267456, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7499}, {'shards': '0,2', 'objects': 2312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9859", 'reported_seq': 9992, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.493240+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:25.493240+0000', 'last_peered': '2024-09-11T05:17:25.493240+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:25.493240+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 9859, 'log_dups_size': 0, 'ondisk_log_size': 9859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40357888, 'num_objects': 9853, 'num_object_clones': 0, 'num_object_copies': 19706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7024, 'num_write_kb': 28096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6277}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9835", 'reported_seq': 9966, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:26.044107+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:26.044107+0000', 'last_peered': '2024-09-11T05:17:26.044107+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:26.044107+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 9835, 'log_dups_size': 0, 'ondisk_log_size': 9835, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40255488, 'num_objects': 9828, 'num_object_clones': 0, 'num_object_copies': 19656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7903, 'num_write_kb': 31612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 6303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9649", 'reported_seq': 9814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.468571+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:25.468571+0000', 'last_peered': '2024-09-11T05:17:25.468571+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:25.468571+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 9649, 'log_dups_size': 0, 'ondisk_log_size': 9649, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39489536, 'num_objects': 9641, 'num_object_clones': 0, 'num_object_copies': 19282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9641, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6402, 'num_write_kb': 25608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6199}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9837", 'reported_seq': 9955, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:26.202050+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:26.202050+0000', 'last_peered': '2024-09-11T05:17:26.202050+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:26.202050+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 9837, 'log_dups_size': 0, 'ondisk_log_size': 9837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40271872, 'num_objects': 9832, 'num_object_clones': 0, 'num_object_copies': 19664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7866, 'num_write_kb': 31464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 6310}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:29.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9866", 'reported_seq': 9997, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.505785+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:25.505785+0000', 'last_peered': '2024-09-11T05:17:25.505785+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:25.505785+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 9866, 'log_dups_size': 0, 'ondisk_log_size': 9866, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40402944, 'num_objects': 9864, 'num_object_clones': 0, 'num_object_copies': 19728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6899, 'num_write_kb': 27596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6241}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:29.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9628", 'reported_seq': 9761, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.480348+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:25.480348+0000', 'last_peered': '2024-09-11T05:17:25.480348+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:25.480348+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 9628, 'log_dups_size': 0, 'ondisk_log_size': 9628, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39403520, 'num_objects': 9620, 'num_object_clones': 0, 'num_object_copies': 19240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6494, 'num_write_kb': 25976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6125}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:29.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9696", 'reported_seq': 9828, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.502309+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:25.502309+0000', 'last_peered': '2024-09-11T05:17:25.502309+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:25.502309+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 9696, 'log_dups_size': 0, 'ondisk_log_size': 9696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39698432, 'num_objects': 9692, 'num_object_clones': 0, 'num_object_copies': 19384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6454, 'num_write_kb': 25816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6148}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:29.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9607", 'reported_seq': 9751, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:25.432525+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:25.432525+0000', 'last_peered': '2024-09-11T05:17:25.432525+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:25.432525+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 9607, 'log_dups_size': 0, 'ondisk_log_size': 9607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39301737, 'num_objects': 9596, 'num_object_clones': 0, 'num_object_copies': 19192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6614, 'num_write_kb': 26456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6081}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:29.826 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:30.168 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 237 20 158018 157998 2.60304 1.64453 0.0397519 0.0300053 2024-09-11T05:17:30.858 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:30.858 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/312168 objects degraded (11.350%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":328,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156084,"data_bytes":639308412,"bytes_used":2122756096,"bytes_avail":287787536384,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":312168,"degraded_ratio":0.1134997821685759,"misplaced_objects":2312,"misplaced_total":312168,"misplaced_ratio":0.0074062684195689513,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":49440,"recovering_keys_per_sec":0,"num_objects_recovered":145,"num_bytes_recovered":593920,"num_keys_recovered":0,"write_bytes_sec":1503003,"read_op_per_sec":0,"write_op_per_sec":366},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:30.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:30.941 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:31.166 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 238 20 158442 158422 2.59903 1.65625 0.0825819 0.0300513 2024-09-11T05:17:31.785 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:31.786 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:31.845 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-11T05:17:31.476236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648417916,"num_objects":158308,"num_object_clones":0,"num_object_copies":316616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":158308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121453,"num_write_kb":485806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4841,"num_bytes_recovered":19816467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158390,"ondisk_log_size":158390,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2092420,"kb_used_data":1281040,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281023100,"statfs":{"total":289910292480,"available":287767654400,"internally_reserved":0,"allocated":1311784960,"data_stored":1310942664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":830786561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,7,10,4,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27164672,"num_objects":6632,"num_object_clones":0,"num_object_copies":13264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-218,"num_objects_unfound":0,"num_objects_dirty":6632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":892928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004485"},"pg_stats":[{"pgid":"2.f","version":"45'10008","reported_seq":10086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.594352+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:30.594352+0000","last_peered":"2024-09-11T05:17:30.594352+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:30.594352+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40992768,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10008,"num_write_kb":40032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6333},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9913","reported_seq":10015,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.195106+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:31.195106+0000","last_peered":"2024-09-11T05:17:31.195106+0000","last_clean":"2024-09-11T05:17:31.195106+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:31.195106+0000","last_undegraded":"2024-09-11T05:17:31.195106+0000","last_fullsized":"2024-09-11T05:17:31.195106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9913,"log_dups_size":0,"ondisk_log_size":9913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8311,"num_write_kb":33244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9813","reported_seq":9938,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.155042+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:31.155042+0000","last_peered":"2024-09-11T05:17:31.155042+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:31.155042+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6315}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9936","reported_seq":10004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.191616+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:31.191616+0000","last_peered":"2024-09-11T05:17:31.191616+0000","last_clean":"2024-09-11T05:17:31.191616+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:31.191616+0000","last_undegraded":"2024-09-11T05:17:31.191616+0000","last_fullsized":"2024-09-11T05:17:31.191616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9936,"log_dups_size":0,"ondisk_log_size":9936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40697856,"num_objects":9936,"num_object_clones":0,"num_object_copies":19872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9936,"num_write_kb":39744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10009","reported_seq":10139,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:30.544504+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:30.544504+0000","last_peered":"2024-09-11T05:17:30.544504+0000","last_clean":"2024-09-11T05:17:30.544504+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:30.544504+0000","last_undegraded":"2024-09-11T05:17:30.544504+0000","last_fullsized":"2024-09-11T05:17:30.544504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968192,"num_objects":10002,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9957","reported_seq":12825,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.571164+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:30.571164+0000","last_peered":"2024-09-11T05:17:30.571164+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:30.571164+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9957,"log_dups_size":0,"ondisk_log_size":9957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40763392,"num_objects":9952,"num_object_clones":0,"num_object_copies":19904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":9952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6501,"num_write_kb":26004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":5566464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7711},{"shards":"0,2","objects":2239}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10025","reported_seq":10158,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.577047+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:30.577047+0000","last_peered":"2024-09-11T05:17:30.577047+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:30.577047+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41037824,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":28760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6443},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9869","reported_seq":9993,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:30.554835+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:30.554835+0000","last_peered":"2024-09-11T05:17:30.554835+0000","last_clean":"2024-09-11T05:17:30.554835+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:30.554835+0000","last_undegraded":"2024-09-11T05:17:30.554835+0000","last_fullsized":"2024-09-11T05:17:30.554835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9869,"log_dups_size":0,"ondisk_log_size":9869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40415232,"num_objects":9867,"num_object_clones":0,"num_object_copies":19734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":28688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9948","reported_seq":10079,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.212286+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:31.212286+0000","last_peered":"2024-09-11T05:17:31.212286+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:31.212286+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9948,"log_dups_size":0,"ondisk_log_size":9948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9780","reported_seq":9945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.530284+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:30.530284+0000","last_peered":"2024-09-11T05:17:30.530284+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:30.530284+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9780,"log_dups_size":0,"ondisk_log_size":9780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40026112,"num_objects":9772,"num_object_clones":0,"num_object_copies":19544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6330},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9964","reported_seq":10082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.145232+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:31.145232+0000","last_peered":"2024-09-11T05:17:31.145232+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:31.145232+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9964,"log_dups_size":0,"ondisk_log_size":9964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40792064,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7993,"num_write_kb":31972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9810","reported_seq":16895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.128354+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:31.128354+0000","last_peered":"2024-09-11T05:17:31.128354+0000","last_clean":"2024-09-11T05:17:31.128354+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:31.128354+0000","last_undegraded":"2024-09-11T05:17:31.128354+0000","last_fullsized":"2024-09-11T05:17:31.128354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9810,"log_dups_size":0,"ondisk_log_size":9810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40157184,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":30852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10011","reported_seq":10142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.601505+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:30.601505+0000","last_peered":"2024-09-11T05:17:30.601505+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:30.601505+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40996864,"num_objects":10009,"num_object_clones":0,"num_object_copies":20018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7044,"num_write_kb":28176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6386},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9766","reported_seq":9899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.561441+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:30.561441+0000","last_peered":"2024-09-11T05:17:30.561441+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:30.561441+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9766,"log_dups_size":0,"ondisk_log_size":9766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6263},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9832","reported_seq":9966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.569719+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:30.569719+0000","last_peered":"2024-09-11T05:17:30.569719+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:30.569719+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9832,"log_dups_size":0,"ondisk_log_size":9832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6590,"num_write_kb":26360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6284},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9746","reported_seq":9890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.547173+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:30.547173+0000","last_peered":"2024-09-11T05:17:30.547173+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:30.547173+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9746,"log_dups_size":0,"ondisk_log_size":9746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39871081,"num_objects":9735,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6220},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648417897,"num_objects":158306,"num_object_clones":0,"num_object_copies":316612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":158306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121451,"num_write_kb":485804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4840,"num_bytes_recovered":19816448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1312231424,"data_stored":1312151407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158387,"ondisk_log_size":158387,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687032,"kb_used_data":430440,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93684808,"statfs":{"total":96636764160,"available":95933243392,"internally_reserved":0,"allocated":440770560,"data_stored":440515706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545480,"kb_used_data":319416,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93826360,"statfs":{"total":96636764160,"available":96078192640,"internally_reserved":0,"allocated":327081984,"data_stored":326753473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":231468715},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215164,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859908,"kb_used_data":531184,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93511932,"statfs":{"total":96636764160,"available":95756218368,"internally_reserved":0,"allocated":543932416,"data_stored":543673485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336588459},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543870976,"data_stored":543867497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327106560,"data_stored":327033501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441253888,"data_stored":441250409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:32.166 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout: 239 20 158873 158853 2.5952 1.68359 0.0662862 0.0300958 2024-09-11T05:17:33.166 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Total time run: 240.021 2024-09-11T05:17:33.166 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Total writes made: 159273 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Write size: 4096 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Object size: 4096 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Bandwidth (MB/sec): 2.59211 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Stddev Bandwidth: 1.66748 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Max bandwidth (MB/sec): 11.0156 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Min bandwidth (MB/sec): 0 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Average IOPS: 663 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Stddev IOPS: 426.878 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Max IOPS: 2820 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Min IOPS: 0 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Average Latency(s): 0.0301342 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Stddev Latency(s): 0.0419138 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Max latency(s): 2.23343 2024-09-11T05:17:33.167 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Min latency(s): 0.00201616 2024-09-11T05:17:33.183 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T05:17:34.847 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:35.633 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:35.681 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-11T05:17:35.480247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648417916,"num_objects":158308,"num_object_clones":0,"num_object_copies":316616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":158308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121453,"num_write_kb":485806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4841,"num_bytes_recovered":19816467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158390,"ondisk_log_size":158390,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107368,"kb_used_data":1287668,"kb_used_omap":12,"kb_used_meta":819635,"kb_avail":281008152,"statfs":{"total":289910292480,"available":287752347648,"internally_reserved":0,"allocated":1318572032,"data_stored":1317729736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":839306241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,7,10,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-136,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":17540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005828"},"pg_stats":[{"pgid":"2.f","version":"45'10008","reported_seq":10086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.594352+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:30.594352+0000","last_peered":"2024-09-11T05:17:30.594352+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:30.594352+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40992768,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10008,"num_write_kb":40032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6333},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9913","reported_seq":10015,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.195106+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:31.195106+0000","last_peered":"2024-09-11T05:17:31.195106+0000","last_clean":"2024-09-11T05:17:31.195106+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:31.195106+0000","last_undegraded":"2024-09-11T05:17:31.195106+0000","last_fullsized":"2024-09-11T05:17:31.195106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9913,"log_dups_size":0,"ondisk_log_size":9913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8311,"num_write_kb":33244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9813","reported_seq":9938,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.155042+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:31.155042+0000","last_peered":"2024-09-11T05:17:31.155042+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:31.155042+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6315}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9936","reported_seq":10004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.191616+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:31.191616+0000","last_peered":"2024-09-11T05:17:31.191616+0000","last_clean":"2024-09-11T05:17:31.191616+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:31.191616+0000","last_undegraded":"2024-09-11T05:17:31.191616+0000","last_fullsized":"2024-09-11T05:17:31.191616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9936,"log_dups_size":0,"ondisk_log_size":9936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40697856,"num_objects":9936,"num_object_clones":0,"num_object_copies":19872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9936,"num_write_kb":39744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10009","reported_seq":10139,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:30.544504+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:30.544504+0000","last_peered":"2024-09-11T05:17:30.544504+0000","last_clean":"2024-09-11T05:17:30.544504+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:30.544504+0000","last_undegraded":"2024-09-11T05:17:30.544504+0000","last_fullsized":"2024-09-11T05:17:30.544504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968192,"num_objects":10002,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9957","reported_seq":12825,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.571164+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:30.571164+0000","last_peered":"2024-09-11T05:17:30.571164+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:30.571164+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9957,"log_dups_size":0,"ondisk_log_size":9957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40763392,"num_objects":9952,"num_object_clones":0,"num_object_copies":19904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":9952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6501,"num_write_kb":26004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":5566464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7711},{"shards":"0,2","objects":2239}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10025","reported_seq":10158,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.577047+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:30.577047+0000","last_peered":"2024-09-11T05:17:30.577047+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:30.577047+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41037824,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":28760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6443},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9869","reported_seq":9993,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:30.554835+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:30.554835+0000","last_peered":"2024-09-11T05:17:30.554835+0000","last_clean":"2024-09-11T05:17:30.554835+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:30.554835+0000","last_undegraded":"2024-09-11T05:17:30.554835+0000","last_fullsized":"2024-09-11T05:17:30.554835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9869,"log_dups_size":0,"ondisk_log_size":9869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40415232,"num_objects":9867,"num_object_clones":0,"num_object_copies":19734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":28688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9948","reported_seq":10079,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.212286+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:31.212286+0000","last_peered":"2024-09-11T05:17:31.212286+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:31.212286+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9948,"log_dups_size":0,"ondisk_log_size":9948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9780","reported_seq":9945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.530284+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:30.530284+0000","last_peered":"2024-09-11T05:17:30.530284+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:30.530284+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9780,"log_dups_size":0,"ondisk_log_size":9780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40026112,"num_objects":9772,"num_object_clones":0,"num_object_copies":19544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6330},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9964","reported_seq":10082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.145232+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:31.145232+0000","last_peered":"2024-09-11T05:17:31.145232+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:31.145232+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9964,"log_dups_size":0,"ondisk_log_size":9964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40792064,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7993,"num_write_kb":31972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9810","reported_seq":16895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.128354+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:31.128354+0000","last_peered":"2024-09-11T05:17:31.128354+0000","last_clean":"2024-09-11T05:17:31.128354+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:31.128354+0000","last_undegraded":"2024-09-11T05:17:31.128354+0000","last_fullsized":"2024-09-11T05:17:31.128354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9810,"log_dups_size":0,"ondisk_log_size":9810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40157184,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":30852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10011","reported_seq":10142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.601505+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:30.601505+0000","last_peered":"2024-09-11T05:17:30.601505+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:30.601505+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40996864,"num_objects":10009,"num_object_clones":0,"num_object_copies":20018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7044,"num_write_kb":28176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6386},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9766","reported_seq":9899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.561441+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:30.561441+0000","last_peered":"2024-09-11T05:17:30.561441+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:30.561441+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9766,"log_dups_size":0,"ondisk_log_size":9766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6263},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9832","reported_seq":9966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.569719+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:30.569719+0000","last_peered":"2024-09-11T05:17:30.569719+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:30.569719+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9832,"log_dups_size":0,"ondisk_log_size":9832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6590,"num_write_kb":26360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6284},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9746","reported_seq":9890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.547173+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:30.547173+0000","last_peered":"2024-09-11T05:17:30.547173+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:30.547173+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9746,"log_dups_size":0,"ondisk_log_size":9746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39871081,"num_objects":9735,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6220},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648417897,"num_objects":158306,"num_object_clones":0,"num_object_copies":316612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":158306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121451,"num_write_kb":485804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4840,"num_bytes_recovered":19816448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1319174144,"data_stored":1319094127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158387,"ondisk_log_size":158387,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687032,"kb_used_data":430440,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93684808,"statfs":{"total":96636764160,"available":95933243392,"internally_reserved":0,"allocated":440770560,"data_stored":440515706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560428,"kb_used_data":326044,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93811412,"statfs":{"total":96636764160,"available":96062885888,"internally_reserved":0,"allocated":333869056,"data_stored":333540545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215164,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859908,"kb_used_data":531184,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93511932,"statfs":{"total":96636764160,"available":95756218368,"internally_reserved":0,"allocated":543932416,"data_stored":543673485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336588459},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543870976,"data_stored":543867497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334049280,"data_stored":333976221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441253888,"data_stored":441250409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:35.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:36.498 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:36.571 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-11T05:17:35.480247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648417916,"num_objects":158308,"num_object_clones":0,"num_object_copies":316616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":158308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121453,"num_write_kb":485806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4841,"num_bytes_recovered":19816467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158390,"ondisk_log_size":158390,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107368,"kb_used_data":1287668,"kb_used_omap":12,"kb_used_meta":819635,"kb_avail":281008152,"statfs":{"total":289910292480,"available":287752347648,"internally_reserved":0,"allocated":1318572032,"data_stored":1317729736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":839306241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,7,10,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-136,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":17540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005828"},"pg_stats":[{"pgid":"2.f","version":"45'10008","reported_seq":10086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.594352+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:30.594352+0000","last_peered":"2024-09-11T05:17:30.594352+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:30.594352+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40992768,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10008,"num_write_kb":40032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6333},{"shards":"0,2","objects":3674}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9913","reported_seq":10015,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.195106+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:31.195106+0000","last_peered":"2024-09-11T05:17:31.195106+0000","last_clean":"2024-09-11T05:17:31.195106+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:31.195106+0000","last_undegraded":"2024-09-11T05:17:31.195106+0000","last_fullsized":"2024-09-11T05:17:31.195106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":9913,"log_dups_size":0,"ondisk_log_size":9913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8311,"num_write_kb":33244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9813","reported_seq":9938,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.155042+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:31.155042+0000","last_peered":"2024-09-11T05:17:31.155042+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:31.155042+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9813,"log_dups_size":0,"ondisk_log_size":9813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":6315}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9936","reported_seq":10004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.191616+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:31.191616+0000","last_peered":"2024-09-11T05:17:31.191616+0000","last_clean":"2024-09-11T05:17:31.191616+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:31.191616+0000","last_undegraded":"2024-09-11T05:17:31.191616+0000","last_fullsized":"2024-09-11T05:17:31.191616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":9936,"log_dups_size":0,"ondisk_log_size":9936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40697856,"num_objects":9936,"num_object_clones":0,"num_object_copies":19872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9936,"num_write_kb":39744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10009","reported_seq":10139,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:30.544504+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:30.544504+0000","last_peered":"2024-09-11T05:17:30.544504+0000","last_clean":"2024-09-11T05:17:30.544504+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:30.544504+0000","last_undegraded":"2024-09-11T05:17:30.544504+0000","last_fullsized":"2024-09-11T05:17:30.544504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968192,"num_objects":10002,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9957","reported_seq":12825,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.571164+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:30.571164+0000","last_peered":"2024-09-11T05:17:30.571164+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:30.571164+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":9957,"log_dups_size":0,"ondisk_log_size":9957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40763392,"num_objects":9952,"num_object_clones":0,"num_object_copies":19904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":9952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6501,"num_write_kb":26004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":5566464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7711},{"shards":"0,2","objects":2239}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10025","reported_seq":10158,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.577047+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:30.577047+0000","last_peered":"2024-09-11T05:17:30.577047+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:30.577047+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41037824,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":28760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6443},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9869","reported_seq":9993,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:30.554835+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:30.554835+0000","last_peered":"2024-09-11T05:17:30.554835+0000","last_clean":"2024-09-11T05:17:30.554835+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:30.554835+0000","last_undegraded":"2024-09-11T05:17:30.554835+0000","last_fullsized":"2024-09-11T05:17:30.554835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":9869,"log_dups_size":0,"ondisk_log_size":9869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40415232,"num_objects":9867,"num_object_clones":0,"num_object_copies":19734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":28688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9948","reported_seq":10079,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.212286+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:31.212286+0000","last_peered":"2024-09-11T05:17:31.212286+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:31.212286+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":9948,"log_dups_size":0,"ondisk_log_size":9948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3525},{"shards":"2","objects":6416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9780","reported_seq":9945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.530284+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:30.530284+0000","last_peered":"2024-09-11T05:17:30.530284+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:30.530284+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9780,"log_dups_size":0,"ondisk_log_size":9780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40026112,"num_objects":9772,"num_object_clones":0,"num_object_copies":19544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6330},{"shards":"0,2","objects":3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9964","reported_seq":10082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:31.145232+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:31.145232+0000","last_peered":"2024-09-11T05:17:31.145232+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:31.145232+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":9964,"log_dups_size":0,"ondisk_log_size":9964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40792064,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7993,"num_write_kb":31972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3522},{"shards":"2","objects":6437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9810","reported_seq":16895,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:31.128354+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:31.128354+0000","last_peered":"2024-09-11T05:17:31.128354+0000","last_clean":"2024-09-11T05:17:31.128354+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:31.128354+0000","last_undegraded":"2024-09-11T05:17:31.128354+0000","last_fullsized":"2024-09-11T05:17:31.128354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9810,"log_dups_size":0,"ondisk_log_size":9810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40157184,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":30852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10011","reported_seq":10142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.601505+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:30.601505+0000","last_peered":"2024-09-11T05:17:30.601505+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:30.601505+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40996864,"num_objects":10009,"num_object_clones":0,"num_object_copies":20018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7044,"num_write_kb":28176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6386},{"shards":"0,2","objects":3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9766","reported_seq":9899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.561441+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:30.561441+0000","last_peered":"2024-09-11T05:17:30.561441+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:30.561441+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9766,"log_dups_size":0,"ondisk_log_size":9766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6263},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9832","reported_seq":9966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.569719+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:30.569719+0000","last_peered":"2024-09-11T05:17:30.569719+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:30.569719+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9832,"log_dups_size":0,"ondisk_log_size":9832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6590,"num_write_kb":26360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6284},{"shards":"0,2","objects":3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9746","reported_seq":9890,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:30.547173+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:30.547173+0000","last_peered":"2024-09-11T05:17:30.547173+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:30.547173+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9746,"log_dups_size":0,"ondisk_log_size":9746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39871081,"num_objects":9735,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6220},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648417897,"num_objects":158306,"num_object_clones":0,"num_object_copies":316612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2239,"num_objects_unfound":0,"num_objects_dirty":158306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121451,"num_write_kb":485804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4840,"num_bytes_recovered":19816448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1319174144,"data_stored":1319094127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158387,"ondisk_log_size":158387,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687032,"kb_used_data":430440,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93684808,"statfs":{"total":96636764160,"available":95933243392,"internally_reserved":0,"allocated":440770560,"data_stored":440515706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560428,"kb_used_data":326044,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93811412,"statfs":{"total":96636764160,"available":96062885888,"internally_reserved":0,"allocated":333869056,"data_stored":333540545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215164,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859908,"kb_used_data":531184,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93511932,"statfs":{"total":96636764160,"available":95756218368,"internally_reserved":0,"allocated":543932416,"data_stored":543673485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336588459},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543870976,"data_stored":543867497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334049280,"data_stored":333976221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441253888,"data_stored":441250409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:36.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:36.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10008", 'reported_seq': 10086, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.594352+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:30.594352+0000', 'last_peered': '2024-09-11T05:17:30.594352+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:30.594352+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40992768, 'num_objects': 10008, 'num_object_clones': 0, 'num_object_copies': 20016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10008, 'num_write_kb': 40032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6333}, {'shards': '0,2', 'objects': 3674}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:36.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9813", 'reported_seq': 9938, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:31.155042+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:31.155042+0000', 'last_peered': '2024-09-11T05:17:31.155042+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:31.155042+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 9813, 'log_dups_size': 0, 'ondisk_log_size': 9813, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40177664, 'num_objects': 9809, 'num_object_clones': 0, 'num_object_copies': 19618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9809, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8216, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 6315}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9957", 'reported_seq': 12825, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.571164+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:30.571164+0000', 'last_peered': '2024-09-11T05:17:30.571164+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:30.571164+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 9957, 'log_dups_size': 0, 'ondisk_log_size': 9957, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40763392, 'num_objects': 9952, 'num_object_clones': 0, 'num_object_copies': 19904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2239, 'num_objects_unfound': 0, 'num_objects_dirty': 9952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6501, 'num_write_kb': 26004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1359, 'num_bytes_recovered': 5566464, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7711}, {'shards': '0,2', 'objects': 2239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10025", 'reported_seq': 10158, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.577047+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:30.577047+0000', 'last_peered': '2024-09-11T05:17:30.577047+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:30.577047+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41037824, 'num_objects': 10019, 'num_object_clones': 0, 'num_object_copies': 20038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7190, 'num_write_kb': 28760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6443}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9948", 'reported_seq': 10079, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:31.212286+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:31.212286+0000', 'last_peered': '2024-09-11T05:17:31.212286+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:31.212286+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 9948, 'log_dups_size': 0, 'ondisk_log_size': 9948, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40718336, 'num_objects': 9941, 'num_object_clones': 0, 'num_object_copies': 19882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8016, 'num_write_kb': 32064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3525}, {'shards': '2', 'objects': 6416}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9780", 'reported_seq': 9945, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.530284+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:30.530284+0000', 'last_peered': '2024-09-11T05:17:30.530284+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:30.530284+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 9780, 'log_dups_size': 0, 'ondisk_log_size': 9780, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40026112, 'num_objects': 9772, 'num_object_clones': 0, 'num_object_copies': 19544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6533, 'num_write_kb': 26132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6330}, {'shards': '0,2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:36.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9964", 'reported_seq': 10082, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:31.145232+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:31.145232+0000', 'last_peered': '2024-09-11T05:17:31.145232+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:31.145232+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 9964, 'log_dups_size': 0, 'ondisk_log_size': 9964, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40792064, 'num_objects': 9959, 'num_object_clones': 0, 'num_object_copies': 19918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9959, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7993, 'num_write_kb': 31972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3522}, {'shards': '2', 'objects': 6437}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10011", 'reported_seq': 10142, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.601505+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:30.601505+0000', 'last_peered': '2024-09-11T05:17:30.601505+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:30.601505+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40996864, 'num_objects': 10009, 'num_object_clones': 0, 'num_object_copies': 20018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7044, 'num_write_kb': 28176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6386}, {'shards': '0,2', 'objects': 3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9766", 'reported_seq': 9899, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.561441+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:30.561441+0000', 'last_peered': '2024-09-11T05:17:30.561441+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:30.561441+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 9766, 'log_dups_size': 0, 'ondisk_log_size': 9766, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39968768, 'num_objects': 9758, 'num_object_clones': 0, 'num_object_copies': 19516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9758, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6632, 'num_write_kb': 26528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6263}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9832", 'reported_seq': 9966, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.569719+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:30.569719+0000', 'last_peered': '2024-09-11T05:17:30.569719+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:30.569719+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 9832, 'log_dups_size': 0, 'ondisk_log_size': 9832, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40255488, 'num_objects': 9828, 'num_object_clones': 0, 'num_object_copies': 19656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6590, 'num_write_kb': 26360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6284}, {'shards': '0,2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:36.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9746", 'reported_seq': 9890, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:30.547173+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:30.547173+0000', 'last_peered': '2024-09-11T05:17:30.547173+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:30.547173+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 9746, 'log_dups_size': 0, 'ondisk_log_size': 9746, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39871081, 'num_objects': 9735, 'num_object_clones': 0, 'num_object_copies': 19470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9735, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6753, 'num_write_kb': 27012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6220}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:36.576 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:37.308 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:37.308 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/316616 objects degraded (11.191%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":335,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158308,"data_bytes":648417916,"bytes_used":2157944832,"bytes_avail":287752347648,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":316616,"degraded_ratio":0.11190527326477499,"misplaced_objects":2239,"misplaced_total":316616,"misplaced_ratio":0.0070716577810344379,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":46398,"recovering_keys_per_sec":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"write_bytes_sec":1496019,"read_op_per_sec":0,"write_op_per_sec":365},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:37.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:37.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:38.159 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:38.161 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:38.250 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-11T05:17:37.483768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647410324,"num_objects":158063,"num_object_clones":0,"num_object_copies":316126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2150,"num_objects_unfound":0,"num_objects_dirty":158063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123636,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4930,"num_bytes_recovered":20181011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159973,"ondisk_log_size":159973,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122212,"kb_used_data":1288628,"kb_used_omap":12,"kb_used_meta":833523,"kb_avail":280993308,"statfs":{"total":289910292480,"available":287737147392,"internally_reserved":0,"allocated":1319555072,"data_stored":1318708704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":853527553},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,8,11,13,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16953368,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-225,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6568,"num_write_kb":21413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":921600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008983"},"pg_stats":[{"pgid":"2.f","version":"45'10169","reported_seq":10248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.492440+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:35.492440+0000","last_peered":"2024-09-11T05:17:35.492440+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:35.492440+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41046016,"num_objects":10021,"num_object_clones":0,"num_object_copies":20042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6346},{"shards":"0,2","objects":3600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10040","reported_seq":10139,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:36.283542+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:36.283542+0000","last_peered":"2024-09-11T05:17:36.283542+0000","last_clean":"2024-09-11T05:17:36.283542+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:36.283542+0000","last_undegraded":"2024-09-11T05:17:36.283542+0000","last_fullsized":"2024-09-11T05:17:36.283542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40386560,"num_objects":9860,"num_object_clones":0,"num_object_copies":19720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8438,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9952","reported_seq":10078,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:36.287505+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:36.287505+0000","last_peered":"2024-09-11T05:17:36.287505+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:36.287505+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":9952,"log_dups_size":0,"ondisk_log_size":9952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39997464,"num_objects":9766,"num_object_clones":0,"num_object_copies":19532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8355,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3403},{"shards":"2","objects":6272}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10053","reported_seq":10124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:36.161149+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:36.161149+0000","last_peered":"2024-09-11T05:17:36.161149+0000","last_clean":"2024-09-11T05:17:36.161149+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:36.161149+0000","last_undegraded":"2024-09-11T05:17:36.161149+0000","last_fullsized":"2024-09-11T05:17:36.161149+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10053,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10147","reported_seq":10277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:35.575359+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:35.575359+0000","last_peered":"2024-09-11T05:17:35.575359+0000","last_clean":"2024-09-11T05:17:35.575359+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:35.575359+0000","last_undegraded":"2024-09-11T05:17:35.575359+0000","last_fullsized":"2024-09-11T05:17:35.575359+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6981,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10105","reported_seq":13151,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.629267+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:35.629267+0000","last_peered":"2024-09-11T05:17:35.629267+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:35.629267+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40673280,"num_objects":9930,"num_object_clones":0,"num_object_copies":19860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2150,"num_objects_unfound":0,"num_objects_dirty":9930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6649,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1448,"num_bytes_recovered":5931008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7778},{"shards":"0,2","objects":2150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10177","reported_seq":10310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.632856+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:35.632856+0000","last_peered":"2024-09-11T05:17:35.632856+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:35.632856+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41062400,"num_objects":10025,"num_object_clones":0,"num_object_copies":20050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7342,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6449},{"shards":"0,2","objects":3503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10013","reported_seq":10137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:35.626829+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:35.626829+0000","last_peered":"2024-09-11T05:17:35.626829+0000","last_clean":"2024-09-11T05:17:35.626829+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:35.626829+0000","last_undegraded":"2024-09-11T05:17:35.626829+0000","last_fullsized":"2024-09-11T05:17:35.626829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40431616,"num_objects":9871,"num_object_clones":0,"num_object_copies":19742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10071","reported_seq":10202,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:36.307036+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:36.307036+0000","last_peered":"2024-09-11T05:17:36.307036+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:36.307036+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3443},{"shards":"2","objects":6375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9904","reported_seq":10066,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.620460+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:35.620460+0000","last_peered":"2024-09-11T05:17:35.620460+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:35.620460+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":9904,"log_dups_size":0,"ondisk_log_size":9904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6657,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6326},{"shards":"0,2","objects":3378}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10109","reported_seq":10227,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:36.162143+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:36.162143+0000","last_peered":"2024-09-11T05:17:36.162143+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:36.162143+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40648704,"num_objects":9924,"num_object_clones":0,"num_object_copies":19848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8138,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3432},{"shards":"2","objects":6402}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9940","reported_seq":17025,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:36.276434+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:36.276434+0000","last_peered":"2024-09-11T05:17:36.276434+0000","last_clean":"2024-09-11T05:17:36.276434+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:36.276434+0000","last_undegraded":"2024-09-11T05:17:36.276434+0000","last_fullsized":"2024-09-11T05:17:36.276434+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":9940,"log_dups_size":0,"ondisk_log_size":9940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7843,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10152","reported_seq":10282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.630297+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:35.630297+0000","last_peered":"2024-09-11T05:17:35.630297+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:35.630297+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927232,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7185,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6369},{"shards":"0,2","objects":3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9896","reported_seq":10030,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.612199+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:35.612199+0000","last_peered":"2024-09-11T05:17:35.612199+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:35.612199+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":9896,"log_dups_size":0,"ondisk_log_size":9896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6762,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6263},{"shards":"0,2","objects":3430}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9974","reported_seq":10109,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.502353+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:35.502353+0000","last_peered":"2024-09-11T05:17:35.502353+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:35.502353+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":9974,"log_dups_size":0,"ondisk_log_size":9974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40247296,"num_objects":9826,"num_object_clones":0,"num_object_copies":19652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6732,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6282},{"shards":"0,2","objects":3471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9868","reported_seq":10012,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:35.583341+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:35.583341+0000","last_peered":"2024-09-11T05:17:35.583341+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:35.583341+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":9868,"log_dups_size":0,"ondisk_log_size":9868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821929,"num_objects":9723,"num_object_clones":0,"num_object_copies":19446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6875,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6208},{"shards":"0,2","objects":3448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647410305,"num_objects":158061,"num_object_clones":0,"num_object_copies":316122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2150,"num_objects_unfound":0,"num_objects_dirty":158061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123634,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4929,"num_bytes_recovered":20180992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317855232,"data_stored":1317771143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159970,"ondisk_log_size":159970,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691656,"kb_used_data":430904,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93680184,"statfs":{"total":96636764160,"available":95928508416,"internally_reserved":0,"allocated":441245696,"data_stored":440986770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":266989227},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,4,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560428,"kb_used_data":326044,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93811412,"statfs":{"total":96636764160,"available":96062885888,"internally_reserved":0,"allocated":333869056,"data_stored":333540545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":239988395},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215165,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870128,"kb_used_data":531680,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93501712,"statfs":{"total":96636764160,"available":95745753088,"internally_reserved":0,"allocated":544440320,"data_stored":544181389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543731712,"data_stored":543728233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334049280,"data_stored":333976221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440074240,"data_stored":440066689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:41.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:42.136 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:42.137 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:42.235 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-11T05:17:41.484763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638390932,"num_objects":155861,"num_object_clones":0,"num_object_copies":311722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":155861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125838,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5015,"num_bytes_recovered":20529171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107704,"kb_used_data":1278276,"kb_used_omap":12,"kb_used_meta":829363,"kb_avail":281007816,"statfs":{"total":289910292480,"available":287752003584,"internally_reserved":0,"allocated":1308954624,"data_stored":1308104184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":849267713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,2,3,14,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-917480,"num_objects":-223,"num_object_clones":0,"num_object_copies":-446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-247,"num_objects_unfound":0,"num_objects_dirty":-223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6609,"num_write_kb":12769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008994"},"pg_stats":[{"pgid":"2.f","version":"45'10306","reported_seq":10385,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.527953+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:40.527953+0000","last_peered":"2024-09-11T05:17:40.527953+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:40.527953+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10306,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6209},{"shards":"0,2","objects":3463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10184","reported_seq":10286,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.278821+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:41.278821+0000","last_peered":"2024-09-11T05:17:41.278821+0000","last_clean":"2024-09-11T05:17:41.278821+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:41.278821+0000","last_undegraded":"2024-09-11T05:17:41.278821+0000","last_fullsized":"2024-09-11T05:17:41.278821+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8582,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10085","reported_seq":10211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.264220+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:41.264220+0000","last_peered":"2024-09-11T05:17:41.264220+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:41.264220+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8488,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3270},{"shards":"2","objects":6139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10181","reported_seq":10250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.320904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:41.320904+0000","last_peered":"2024-09-11T05:17:41.320904+0000","last_clean":"2024-09-11T05:17:41.320904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:41.320904+0000","last_undegraded":"2024-09-11T05:17:41.320904+0000","last_fullsized":"2024-09-11T05:17:41.320904+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40054784,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10181,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10301","reported_seq":10431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:40.640445+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:40.640445+0000","last_peered":"2024-09-11T05:17:40.640445+0000","last_clean":"2024-09-11T05:17:40.640445+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:40.640445+0000","last_undegraded":"2024-09-11T05:17:40.640445+0000","last_fullsized":"2024-09-11T05:17:40.640445+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7135,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10231","reported_seq":13447,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.637319+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:40.637319+0000","last_peered":"2024-09-11T05:17:40.637319+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:40.637319+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40157184,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6775,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6279168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7737},{"shards":"0,2","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10337","reported_seq":10470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.632875+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:40.632875+0000","last_peered":"2024-09-11T05:17:40.632875+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:40.632875+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7502,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6289},{"shards":"0,2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10143","reported_seq":10267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:40.570382+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:40.570382+0000","last_peered":"2024-09-11T05:17:40.570382+0000","last_clean":"2024-09-11T05:17:40.570382+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:40.570382+0000","last_undegraded":"2024-09-11T05:17:40.570382+0000","last_fullsized":"2024-09-11T05:17:40.570382+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10216","reported_seq":10347,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.332599+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:41.332599+0000","last_peered":"2024-09-11T05:17:41.332599+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:41.332599+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3298},{"shards":"2","objects":6230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10042","reported_seq":10207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.656452+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:40.656452+0000","last_peered":"2024-09-11T05:17:40.656452+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:40.656452+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39444480,"num_objects":9630,"num_object_clones":0,"num_object_copies":19260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6795,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":3240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10228","reported_seq":10346,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.149004+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:41.149004+0000","last_peered":"2024-09-11T05:17:41.149004+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:41.149004+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40161280,"num_objects":9805,"num_object_clones":0,"num_object_copies":19610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3313},{"shards":"2","objects":6283}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10067","reported_seq":17152,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.295325+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:41.295325+0000","last_peered":"2024-09-11T05:17:41.295325+0000","last_clean":"2024-09-11T05:17:41.295325+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:41.295325+0000","last_undegraded":"2024-09-11T05:17:41.295325+0000","last_fullsized":"2024-09-11T05:17:41.295325+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39489536,"num_objects":9641,"num_object_clones":0,"num_object_copies":19282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10283","reported_seq":10415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.369524+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:40.369524+0000","last_peered":"2024-09-11T05:17:40.369524+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:40.369524+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40390656,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":3413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10019","reported_seq":10153,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.625014+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:40.625014+0000","last_peered":"2024-09-11T05:17:40.625014+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:40.625014+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39464960,"num_objects":9635,"num_object_clones":0,"num_object_copies":19270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6885,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6140},{"shards":"0,2","objects":3307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10121","reported_seq":10256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.522414+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:40.522414+0000","last_peered":"2024-09-11T05:17:40.522414+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:40.522414+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6879,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6135},{"shards":"0,2","objects":3324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10028","reported_seq":10172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.605351+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:40.605351+0000","last_peered":"2024-09-11T05:17:40.605351+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:40.605351+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39166569,"num_objects":9563,"num_object_clones":0,"num_object_copies":19126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6048},{"shards":"0,2","objects":3288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638390913,"num_objects":155859,"num_object_clones":0,"num_object_copies":311718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":155859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125836,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5014,"num_bytes_recovered":20529152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1306615808,"data_stored":1306527647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674900,"kb_used_data":426628,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93696940,"statfs":{"total":96636764160,"available":95945666560,"internally_reserved":0,"allocated":436867072,"data_stored":436608146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565252,"kb_used_data":326708,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93806588,"statfs":{"total":96636764160,"available":96057946112,"internally_reserved":0,"allocated":334548992,"data_stored":334216409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215166,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":867552,"kb_used_data":524940,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93504288,"statfs":{"total":96636764160,"available":95748390912,"internally_reserved":0,"allocated":537538560,"data_stored":537279629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,1,9,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536825856,"data_stored":536822377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334147584,"data_stored":334070453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435642368,"data_stored":435634817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:42.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:43.074 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:43.075 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:43.149 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-11T05:17:41.484763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638390932,"num_objects":155861,"num_object_clones":0,"num_object_copies":311722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":155861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125838,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5015,"num_bytes_recovered":20529171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107704,"kb_used_data":1278276,"kb_used_omap":12,"kb_used_meta":829363,"kb_avail":281007816,"statfs":{"total":289910292480,"available":287752003584,"internally_reserved":0,"allocated":1308954624,"data_stored":1308104184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":849267713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,2,3,14,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-917480,"num_objects":-223,"num_object_clones":0,"num_object_copies":-446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-247,"num_objects_unfound":0,"num_objects_dirty":-223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6609,"num_write_kb":12769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008994"},"pg_stats":[{"pgid":"2.f","version":"45'10306","reported_seq":10385,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.527953+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:40.527953+0000","last_peered":"2024-09-11T05:17:40.527953+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:40.527953+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10306,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6209},{"shards":"0,2","objects":3463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10184","reported_seq":10286,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.278821+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:41.278821+0000","last_peered":"2024-09-11T05:17:41.278821+0000","last_clean":"2024-09-11T05:17:41.278821+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:41.278821+0000","last_undegraded":"2024-09-11T05:17:41.278821+0000","last_fullsized":"2024-09-11T05:17:41.278821+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8582,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10085","reported_seq":10211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.264220+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:41.264220+0000","last_peered":"2024-09-11T05:17:41.264220+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:41.264220+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8488,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3270},{"shards":"2","objects":6139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10181","reported_seq":10250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.320904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:41.320904+0000","last_peered":"2024-09-11T05:17:41.320904+0000","last_clean":"2024-09-11T05:17:41.320904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:41.320904+0000","last_undegraded":"2024-09-11T05:17:41.320904+0000","last_fullsized":"2024-09-11T05:17:41.320904+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40054784,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10181,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10301","reported_seq":10431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:40.640445+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:40.640445+0000","last_peered":"2024-09-11T05:17:40.640445+0000","last_clean":"2024-09-11T05:17:40.640445+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:40.640445+0000","last_undegraded":"2024-09-11T05:17:40.640445+0000","last_fullsized":"2024-09-11T05:17:40.640445+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7135,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10231","reported_seq":13447,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.637319+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:40.637319+0000","last_peered":"2024-09-11T05:17:40.637319+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:40.637319+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40157184,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6775,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6279168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7737},{"shards":"0,2","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10337","reported_seq":10470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.632875+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:40.632875+0000","last_peered":"2024-09-11T05:17:40.632875+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:40.632875+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7502,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6289},{"shards":"0,2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10143","reported_seq":10267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:40.570382+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:40.570382+0000","last_peered":"2024-09-11T05:17:40.570382+0000","last_clean":"2024-09-11T05:17:40.570382+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:40.570382+0000","last_undegraded":"2024-09-11T05:17:40.570382+0000","last_fullsized":"2024-09-11T05:17:40.570382+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10216","reported_seq":10347,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.332599+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:41.332599+0000","last_peered":"2024-09-11T05:17:41.332599+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:41.332599+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3298},{"shards":"2","objects":6230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10042","reported_seq":10207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.656452+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:40.656452+0000","last_peered":"2024-09-11T05:17:40.656452+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:40.656452+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39444480,"num_objects":9630,"num_object_clones":0,"num_object_copies":19260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6795,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":3240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10228","reported_seq":10346,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.149004+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:41.149004+0000","last_peered":"2024-09-11T05:17:41.149004+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:41.149004+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40161280,"num_objects":9805,"num_object_clones":0,"num_object_copies":19610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3313},{"shards":"2","objects":6283}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10067","reported_seq":17152,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.295325+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:41.295325+0000","last_peered":"2024-09-11T05:17:41.295325+0000","last_clean":"2024-09-11T05:17:41.295325+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:41.295325+0000","last_undegraded":"2024-09-11T05:17:41.295325+0000","last_fullsized":"2024-09-11T05:17:41.295325+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39489536,"num_objects":9641,"num_object_clones":0,"num_object_copies":19282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10283","reported_seq":10415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.369524+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:40.369524+0000","last_peered":"2024-09-11T05:17:40.369524+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:40.369524+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40390656,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":3413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10019","reported_seq":10153,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.625014+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:40.625014+0000","last_peered":"2024-09-11T05:17:40.625014+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:40.625014+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39464960,"num_objects":9635,"num_object_clones":0,"num_object_copies":19270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6885,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6140},{"shards":"0,2","objects":3307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10121","reported_seq":10256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.522414+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:40.522414+0000","last_peered":"2024-09-11T05:17:40.522414+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:40.522414+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6879,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6135},{"shards":"0,2","objects":3324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10028","reported_seq":10172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.605351+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:40.605351+0000","last_peered":"2024-09-11T05:17:40.605351+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:40.605351+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39166569,"num_objects":9563,"num_object_clones":0,"num_object_copies":19126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6048},{"shards":"0,2","objects":3288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638390913,"num_objects":155859,"num_object_clones":0,"num_object_copies":311718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":155859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125836,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5014,"num_bytes_recovered":20529152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1306615808,"data_stored":1306527647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674900,"kb_used_data":426628,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93696940,"statfs":{"total":96636764160,"available":95945666560,"internally_reserved":0,"allocated":436867072,"data_stored":436608146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565252,"kb_used_data":326708,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93806588,"statfs":{"total":96636764160,"available":96057946112,"internally_reserved":0,"allocated":334548992,"data_stored":334216409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":244248235},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215166,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":867552,"kb_used_data":524940,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93504288,"statfs":{"total":96636764160,"available":95748390912,"internally_reserved":0,"allocated":537538560,"data_stored":537279629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,1,9,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536825856,"data_stored":536822377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334147584,"data_stored":334070453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435642368,"data_stored":435634817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10306", 'reported_seq': 10385, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.527953+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:40.527953+0000', 'last_peered': '2024-09-11T05:17:40.527953+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:40.527953+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40484864, 'num_objects': 9884, 'num_object_clones': 0, 'num_object_copies': 19768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9884, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10306, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6209}, {'shards': '0,2', 'objects': 3463}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10085", 'reported_seq': 10211, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:41.264220+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:41.264220+0000', 'last_peered': '2024-09-11T05:17:41.264220+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:41.264220+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39452696, 'num_objects': 9633, 'num_object_clones': 0, 'num_object_copies': 19266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9633, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8488, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3270}, {'shards': '2', 'objects': 6139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10231", 'reported_seq': 13447, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.637319+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:40.637319+0000', 'last_peered': '2024-09-11T05:17:40.637319+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:40.637319+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40157184, 'num_objects': 9804, 'num_object_clones': 0, 'num_object_copies': 19608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2065, 'num_objects_unfound': 0, 'num_objects_dirty': 9804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6775, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1533, 'num_bytes_recovered': 6279168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7737}, {'shards': '0,2', 'objects': 2065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:43.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10337", 'reported_seq': 10470, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.632875+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:40.632875+0000', 'last_peered': '2024-09-11T05:17:40.632875+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:40.632875+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40407040, 'num_objects': 9865, 'num_object_clones': 0, 'num_object_copies': 19730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7502, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6289}, {'shards': '0,2', 'objects': 3343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10216", 'reported_seq': 10347, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:41.332599+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:41.332599+0000', 'last_peered': '2024-09-11T05:17:41.332599+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:41.332599+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39956480, 'num_objects': 9755, 'num_object_clones': 0, 'num_object_copies': 19510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8284, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3298}, {'shards': '2', 'objects': 6230}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10042", 'reported_seq': 10207, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.656452+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:40.656452+0000', 'last_peered': '2024-09-11T05:17:40.656452+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:40.656452+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39444480, 'num_objects': 9630, 'num_object_clones': 0, 'num_object_copies': 19260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6795, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6188}, {'shards': '0,2', 'objects': 3240}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10228", 'reported_seq': 10346, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:41.149004+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:41.149004+0000', 'last_peered': '2024-09-11T05:17:41.149004+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:41.149004+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40161280, 'num_objects': 9805, 'num_object_clones': 0, 'num_object_copies': 19610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8257, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3313}, {'shards': '2', 'objects': 6283}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10283", 'reported_seq': 10415, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.369524+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:40.369524+0000', 'last_peered': '2024-09-11T05:17:40.369524+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:40.369524+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40390656, 'num_objects': 9861, 'num_object_clones': 0, 'num_object_copies': 19722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7316, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6238}, {'shards': '0,2', 'objects': 3413}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:43.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10019", 'reported_seq': 10153, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.625014+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:40.625014+0000', 'last_peered': '2024-09-11T05:17:40.625014+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:40.625014+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39464960, 'num_objects': 9635, 'num_object_clones': 0, 'num_object_copies': 19270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6885, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6140}, {'shards': '0,2', 'objects': 3307}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:43.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10121", 'reported_seq': 10256, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.522414+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:40.522414+0000', 'last_peered': '2024-09-11T05:17:40.522414+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:40.522414+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39645184, 'num_objects': 9679, 'num_object_clones': 0, 'num_object_copies': 19358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6879, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6135}, {'shards': '0,2', 'objects': 3324}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:43.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10028", 'reported_seq': 10172, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:40.605351+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:40.605351+0000', 'last_peered': '2024-09-11T05:17:40.605351+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:40.605351+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39166569, 'num_objects': 9563, 'num_object_clones': 0, 'num_object_copies': 19126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7035, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6048}, {'shards': '0,2', 'objects': 3288}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:43.153 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:44.122 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:44.123 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/311722 objects degraded (11.366%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":341,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":155861,"data_bytes":638390932,"bytes_used":2148986880,"bytes_avail":287761305600,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":311722,"degraded_ratio":0.11366217334676412,"misplaced_objects":2065,"misplaced_total":311722,"misplaced_ratio":0.0066244923361200051,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59347,"recovering_keys_per_sec":0,"num_objects_recovered":174,"num_bytes_recovered":712704,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":330247,"read_op_per_sec":0,"write_op_per_sec":365},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:44.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:44.173 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:45.125 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:45.126 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:45.193 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-11T05:17:43.485278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638390932,"num_objects":155861,"num_object_clones":0,"num_object_copies":311722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":155861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125838,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5015,"num_bytes_recovered":20529171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2098620,"kb_used_data":1277512,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281016900,"statfs":{"total":289910292480,"available":287761305600,"internally_reserved":0,"allocated":1308172288,"data_stored":1307321848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":840748033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,3,2,6,15,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10026984,"num_objects":-2447,"num_object_clones":0,"num_object_copies":-4894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-174,"num_objects_unfound":0,"num_objects_dirty":-2447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4385,"num_write_kb":3873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":712704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009041"},"pg_stats":[{"pgid":"2.f","version":"45'10306","reported_seq":10385,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.527953+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:40.527953+0000","last_peered":"2024-09-11T05:17:40.527953+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:40.527953+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10306,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6209},{"shards":"0,2","objects":3463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10184","reported_seq":10286,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.278821+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:41.278821+0000","last_peered":"2024-09-11T05:17:41.278821+0000","last_clean":"2024-09-11T05:17:41.278821+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:41.278821+0000","last_undegraded":"2024-09-11T05:17:41.278821+0000","last_fullsized":"2024-09-11T05:17:41.278821+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8582,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10085","reported_seq":10211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.264220+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:41.264220+0000","last_peered":"2024-09-11T05:17:41.264220+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:41.264220+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8488,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3270},{"shards":"2","objects":6139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10181","reported_seq":10250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.320904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:41.320904+0000","last_peered":"2024-09-11T05:17:41.320904+0000","last_clean":"2024-09-11T05:17:41.320904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:41.320904+0000","last_undegraded":"2024-09-11T05:17:41.320904+0000","last_fullsized":"2024-09-11T05:17:41.320904+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40054784,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10181,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10301","reported_seq":10431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:40.640445+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:40.640445+0000","last_peered":"2024-09-11T05:17:40.640445+0000","last_clean":"2024-09-11T05:17:40.640445+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:40.640445+0000","last_undegraded":"2024-09-11T05:17:40.640445+0000","last_fullsized":"2024-09-11T05:17:40.640445+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7135,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10231","reported_seq":13447,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.637319+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:40.637319+0000","last_peered":"2024-09-11T05:17:40.637319+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:40.637319+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40157184,"num_objects":9804,"num_object_clones":0,"num_object_copies":19608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":9804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6775,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6279168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7737},{"shards":"0,2","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10337","reported_seq":10470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.632875+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:40.632875+0000","last_peered":"2024-09-11T05:17:40.632875+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:40.632875+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7502,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6289},{"shards":"0,2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10143","reported_seq":10267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:40.570382+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:40.570382+0000","last_peered":"2024-09-11T05:17:40.570382+0000","last_clean":"2024-09-11T05:17:40.570382+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:40.570382+0000","last_undegraded":"2024-09-11T05:17:40.570382+0000","last_fullsized":"2024-09-11T05:17:40.570382+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10216","reported_seq":10347,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.332599+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:41.332599+0000","last_peered":"2024-09-11T05:17:41.332599+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:41.332599+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3298},{"shards":"2","objects":6230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10042","reported_seq":10207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.656452+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:40.656452+0000","last_peered":"2024-09-11T05:17:40.656452+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:40.656452+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39444480,"num_objects":9630,"num_object_clones":0,"num_object_copies":19260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6795,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":3240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10228","reported_seq":10346,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:41.149004+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:41.149004+0000","last_peered":"2024-09-11T05:17:41.149004+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:41.149004+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40161280,"num_objects":9805,"num_object_clones":0,"num_object_copies":19610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3313},{"shards":"2","objects":6283}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10067","reported_seq":17152,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:41.295325+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:41.295325+0000","last_peered":"2024-09-11T05:17:41.295325+0000","last_clean":"2024-09-11T05:17:41.295325+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:41.295325+0000","last_undegraded":"2024-09-11T05:17:41.295325+0000","last_fullsized":"2024-09-11T05:17:41.295325+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39489536,"num_objects":9641,"num_object_clones":0,"num_object_copies":19282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10283","reported_seq":10415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.369524+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:40.369524+0000","last_peered":"2024-09-11T05:17:40.369524+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:40.369524+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40390656,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":3413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10019","reported_seq":10153,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.625014+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:40.625014+0000","last_peered":"2024-09-11T05:17:40.625014+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:40.625014+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39464960,"num_objects":9635,"num_object_clones":0,"num_object_copies":19270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6885,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6140},{"shards":"0,2","objects":3307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10121","reported_seq":10256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.522414+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:40.522414+0000","last_peered":"2024-09-11T05:17:40.522414+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:40.522414+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6879,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6135},{"shards":"0,2","objects":3324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10028","reported_seq":10172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:40.605351+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:40.605351+0000","last_peered":"2024-09-11T05:17:40.605351+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:40.605351+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39166569,"num_objects":9563,"num_object_clones":0,"num_object_copies":19126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6048},{"shards":"0,2","objects":3288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638390913,"num_objects":155859,"num_object_clones":0,"num_object_copies":311718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":2065,"num_objects_unfound":0,"num_objects_dirty":155859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125836,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5014,"num_bytes_recovered":20529152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1305780224,"data_stored":1305692063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674900,"kb_used_data":426628,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93696940,"statfs":{"total":96636764160,"available":95945666560,"internally_reserved":0,"allocated":436867072,"data_stored":436608146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556168,"kb_used_data":325944,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93815672,"statfs":{"total":96636764160,"available":96067248128,"internally_reserved":0,"allocated":333766656,"data_stored":333434073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215166,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":867552,"kb_used_data":524940,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93504288,"statfs":{"total":96636764160,"available":95748390912,"internally_reserved":0,"allocated":537538560,"data_stored":537279629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,1,9,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536825856,"data_stored":536822377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333312000,"data_stored":333234869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435642368,"data_stored":435634817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:48.195 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:49.149 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:49.149 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:49.255 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-11T05:17:47.488639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629154452,"num_objects":153606,"num_object_clones":0,"num_object_copies":307212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1990,"num_objects_unfound":0,"num_objects_dirty":153606,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128093,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5090,"num_bytes_recovered":20836371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160830,"ondisk_log_size":160830,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2095820,"kb_used_data":1266396,"kb_used_omap":12,"kb_used_meta":829363,"kb_avail":281019700,"statfs":{"total":289910292480,"available":287764172800,"internally_reserved":0,"allocated":1296789504,"data_stored":1295939064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":849267713},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,2,11,5,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19263464,"num_objects":-4702,"num_object_clones":0,"num_object_copies":-9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-4702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6640,"num_write_kb":3873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008392"},"pg_stats":[{"pgid":"2.f","version":"45'10459","reported_seq":10538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.677842+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:45.677842+0000","last_peered":"2024-09-11T05:17:45.677842+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:45.677842+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858176,"num_objects":9731,"num_object_clones":0,"num_object_copies":19462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10459,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6056},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10328","reported_seq":10431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:46.339296+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:46.339296+0000","last_peered":"2024-09-11T05:17:46.339296+0000","last_clean":"2024-09-11T05:17:46.339296+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:46.339296+0000","last_undegraded":"2024-09-11T05:17:46.339296+0000","last_fullsized":"2024-09-11T05:17:46.339296+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39206912,"num_objects":9572,"num_object_clones":0,"num_object_copies":19144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8726,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10225","reported_seq":10349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:46.329353+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:46.329353+0000","last_peered":"2024-09-11T05:17:46.329353+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:46.329353+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38879256,"num_objects":9493,"num_object_clones":0,"num_object_copies":18986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8628,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3130},{"shards":"2","objects":5999}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10314","reported_seq":10386,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:46.329264+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:46.329264+0000","last_peered":"2024-09-11T05:17:46.329264+0000","last_clean":"2024-09-11T05:17:46.329264+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:46.329264+0000","last_undegraded":"2024-09-11T05:17:46.329264+0000","last_fullsized":"2024-09-11T05:17:46.329264+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39510016,"num_objects":9646,"num_object_clones":0,"num_object_copies":19292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10314,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10426","reported_seq":10556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:45.696616+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:45.696616+0000","last_peered":"2024-09-11T05:17:45.696616+0000","last_clean":"2024-09-11T05:17:45.696616+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:45.696616+0000","last_undegraded":"2024-09-11T05:17:45.696616+0000","last_fullsized":"2024-09-11T05:17:45.696616+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39907328,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10374","reported_seq":13740,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.747854+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:45.747854+0000","last_peered":"2024-09-11T05:17:45.747854+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:45.747854+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571456,"num_objects":9661,"num_object_clones":0,"num_object_copies":19322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1990,"num_objects_unfound":0,"num_objects_dirty":9661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6918,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1608,"num_bytes_recovered":6586368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7669},{"shards":"0,2","objects":1990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10477","reported_seq":10606,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.667868+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:45.667868+0000","last_peered":"2024-09-11T05:17:45.667868+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:45.667868+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39833600,"num_objects":9725,"num_object_clones":0,"num_object_copies":19450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7642,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6149},{"shards":"0,2","objects":3203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10326","reported_seq":10450,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:45.666688+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:45.666688+0000","last_peered":"2024-09-11T05:17:45.666688+0000","last_clean":"2024-09-11T05:17:45.666688+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:45.666688+0000","last_undegraded":"2024-09-11T05:17:45.666688+0000","last_fullsized":"2024-09-11T05:17:45.666688+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39149568,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7629,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10362","reported_seq":10493,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:46.306563+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:46.306563+0000","last_peered":"2024-09-11T05:17:46.306563+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:46.306563+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39358464,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8430,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3152},{"shards":"2","objects":6084}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10190","reported_seq":10355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.730106+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:45.730106+0000","last_peered":"2024-09-11T05:17:45.730106+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:45.730106+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38838272,"num_objects":9482,"num_object_clones":0,"num_object_copies":18964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6943,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6040},{"shards":"0,2","objects":3092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10356","reported_seq":10474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:46.295116+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:46.295116+0000","last_peered":"2024-09-11T05:17:46.295116+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:46.295116+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3185},{"shards":"2","objects":6155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10198","reported_seq":17283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:46.252661+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:46.252661+0000","last_peered":"2024-09-11T05:17:46.252661+0000","last_clean":"2024-09-11T05:17:46.252661+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:46.252661+0000","last_undegraded":"2024-09-11T05:17:46.252661+0000","last_fullsized":"2024-09-11T05:17:46.252661+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952960,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8101,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10428","reported_seq":10560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.754109+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:45.754109+0000","last_peered":"2024-09-11T05:17:45.754109+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:45.754109+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7461,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6093},{"shards":"0,2","objects":3268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10162","reported_seq":10293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.717467+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:45.717467+0000","last_peered":"2024-09-11T05:17:45.717467+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:45.717467+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38879232,"num_objects":9492,"num_object_clones":0,"num_object_copies":18984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7028,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5997},{"shards":"0,2","objects":3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10255","reported_seq":10390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.725539+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:45.725539+0000","last_peered":"2024-09-11T05:17:45.725539+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:45.725539+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7013,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6001},{"shards":"0,2","objects":3190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10147","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.766740+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:45.766740+0000","last_peered":"2024-09-11T05:17:45.766740+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:45.766740+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38679145,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7154,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5929},{"shards":"0,2","objects":3169}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629154433,"num_objects":153604,"num_object_clones":0,"num_object_copies":307208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1990,"num_objects_unfound":0,"num_objects_dirty":153604,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128091,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5089,"num_bytes_recovered":20836352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1294245888,"data_stored":1294157727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674508,"kb_used_data":422076,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93697332,"statfs":{"total":96636764160,"available":95946067968,"internally_reserved":0,"allocated":432205824,"data_stored":431946898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556168,"kb_used_data":325944,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93815672,"statfs":{"total":96636764160,"available":96067248128,"internally_reserved":0,"allocated":333766656,"data_stored":333434073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":235728555},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215167,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865144,"kb_used_data":518376,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93506696,"statfs":{"total":96636764160,"available":95750856704,"internally_reserved":0,"allocated":530817024,"data_stored":530558093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,5,2,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529858560,"data_stored":529855081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333312000,"data_stored":333234869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431075328,"data_stored":431067777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:49.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:50.157 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:50.158 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:50.226 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-11T05:17:49.489168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629154452,"num_objects":153606,"num_object_clones":0,"num_object_copies":307212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1990,"num_objects_unfound":0,"num_objects_dirty":153606,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128093,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5090,"num_bytes_recovered":20836371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160830,"ondisk_log_size":160830,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117392,"kb_used_data":1265760,"kb_used_omap":12,"kb_used_meta":851571,"kb_avail":280998128,"statfs":{"total":289910292480,"available":287742083072,"internally_reserved":0,"allocated":1296138240,"data_stored":1295287800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":872008705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,7,5,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18255872,"num_objects":-4457,"num_object_clones":0,"num_object_copies":-8914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-160,"num_objects_unfound":0,"num_objects_dirty":-4457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4457,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005399"},"pg_stats":[{"pgid":"2.f","version":"45'10459","reported_seq":10538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.677842+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:45.677842+0000","last_peered":"2024-09-11T05:17:45.677842+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:45.677842+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858176,"num_objects":9731,"num_object_clones":0,"num_object_copies":19462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10459,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6056},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10328","reported_seq":10431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:46.339296+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:46.339296+0000","last_peered":"2024-09-11T05:17:46.339296+0000","last_clean":"2024-09-11T05:17:46.339296+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:46.339296+0000","last_undegraded":"2024-09-11T05:17:46.339296+0000","last_fullsized":"2024-09-11T05:17:46.339296+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39206912,"num_objects":9572,"num_object_clones":0,"num_object_copies":19144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8726,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10225","reported_seq":10349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:46.329353+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:46.329353+0000","last_peered":"2024-09-11T05:17:46.329353+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:46.329353+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38879256,"num_objects":9493,"num_object_clones":0,"num_object_copies":18986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8628,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3130},{"shards":"2","objects":5999}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10314","reported_seq":10386,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:46.329264+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:46.329264+0000","last_peered":"2024-09-11T05:17:46.329264+0000","last_clean":"2024-09-11T05:17:46.329264+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:46.329264+0000","last_undegraded":"2024-09-11T05:17:46.329264+0000","last_fullsized":"2024-09-11T05:17:46.329264+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39510016,"num_objects":9646,"num_object_clones":0,"num_object_copies":19292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10314,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10426","reported_seq":10556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:45.696616+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:45.696616+0000","last_peered":"2024-09-11T05:17:45.696616+0000","last_clean":"2024-09-11T05:17:45.696616+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:45.696616+0000","last_undegraded":"2024-09-11T05:17:45.696616+0000","last_fullsized":"2024-09-11T05:17:45.696616+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39907328,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10374","reported_seq":13740,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.747854+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:45.747854+0000","last_peered":"2024-09-11T05:17:45.747854+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:45.747854+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571456,"num_objects":9661,"num_object_clones":0,"num_object_copies":19322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1990,"num_objects_unfound":0,"num_objects_dirty":9661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6918,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1608,"num_bytes_recovered":6586368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7669},{"shards":"0,2","objects":1990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10477","reported_seq":10606,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.667868+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:45.667868+0000","last_peered":"2024-09-11T05:17:45.667868+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:45.667868+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39833600,"num_objects":9725,"num_object_clones":0,"num_object_copies":19450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7642,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6149},{"shards":"0,2","objects":3203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10326","reported_seq":10450,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:45.666688+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:45.666688+0000","last_peered":"2024-09-11T05:17:45.666688+0000","last_clean":"2024-09-11T05:17:45.666688+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:45.666688+0000","last_undegraded":"2024-09-11T05:17:45.666688+0000","last_fullsized":"2024-09-11T05:17:45.666688+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39149568,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7629,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10362","reported_seq":10493,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:46.306563+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:46.306563+0000","last_peered":"2024-09-11T05:17:46.306563+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:46.306563+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39358464,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8430,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3152},{"shards":"2","objects":6084}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10190","reported_seq":10355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.730106+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:45.730106+0000","last_peered":"2024-09-11T05:17:45.730106+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:45.730106+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38838272,"num_objects":9482,"num_object_clones":0,"num_object_copies":18964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6943,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6040},{"shards":"0,2","objects":3092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10356","reported_seq":10474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:46.295116+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:46.295116+0000","last_peered":"2024-09-11T05:17:46.295116+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:46.295116+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3185},{"shards":"2","objects":6155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10198","reported_seq":17283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:46.252661+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:46.252661+0000","last_peered":"2024-09-11T05:17:46.252661+0000","last_clean":"2024-09-11T05:17:46.252661+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:46.252661+0000","last_undegraded":"2024-09-11T05:17:46.252661+0000","last_fullsized":"2024-09-11T05:17:46.252661+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952960,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8101,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10428","reported_seq":10560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.754109+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:45.754109+0000","last_peered":"2024-09-11T05:17:45.754109+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:45.754109+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39796736,"num_objects":9716,"num_object_clones":0,"num_object_copies":19432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7461,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6093},{"shards":"0,2","objects":3268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10162","reported_seq":10293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.717467+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:45.717467+0000","last_peered":"2024-09-11T05:17:45.717467+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:45.717467+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38879232,"num_objects":9492,"num_object_clones":0,"num_object_copies":18984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7028,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5997},{"shards":"0,2","objects":3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10255","reported_seq":10390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.725539+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:45.725539+0000","last_peered":"2024-09-11T05:17:45.725539+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:45.725539+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7013,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6001},{"shards":"0,2","objects":3190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10147","reported_seq":10291,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:45.766740+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:45.766740+0000","last_peered":"2024-09-11T05:17:45.766740+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:45.766740+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38679145,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7154,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5929},{"shards":"0,2","objects":3169}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629154433,"num_objects":153604,"num_object_clones":0,"num_object_copies":307208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1990,"num_objects_unfound":0,"num_objects_dirty":153604,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128091,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5089,"num_bytes_recovered":20836352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1293537280,"data_stored":1293449119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674508,"kb_used_data":422076,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93697332,"statfs":{"total":96636764160,"available":95946067968,"internally_reserved":0,"allocated":432205824,"data_stored":431946898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577740,"kb_used_data":325308,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93794100,"statfs":{"total":96636764160,"available":96045158400,"internally_reserved":0,"allocated":333115392,"data_stored":332782809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215167,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865144,"kb_used_data":518376,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93506696,"statfs":{"total":96636764160,"available":95750856704,"internally_reserved":0,"allocated":530817024,"data_stored":530558093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,5,2,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529858560,"data_stored":529855081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332603392,"data_stored":332526261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431075328,"data_stored":431067777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10459", 'reported_seq': 10538, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.677842+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:45.677842+0000', 'last_peered': '2024-09-11T05:17:45.677842+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:45.677842+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39858176, 'num_objects': 9731, 'num_object_clones': 0, 'num_object_copies': 19462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10459, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6056}, {'shards': '0,2', 'objects': 3310}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10225", 'reported_seq': 10349, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:46.329353+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:46.329353+0000', 'last_peered': '2024-09-11T05:17:46.329353+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:46.329353+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38879256, 'num_objects': 9493, 'num_object_clones': 0, 'num_object_copies': 18986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9493, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8628, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3130}, {'shards': '2', 'objects': 5999}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10374", 'reported_seq': 13740, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.747854+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:45.747854+0000', 'last_peered': '2024-09-11T05:17:45.747854+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:45.747854+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39571456, 'num_objects': 9661, 'num_object_clones': 0, 'num_object_copies': 19322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1990, 'num_objects_unfound': 0, 'num_objects_dirty': 9661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6918, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1608, 'num_bytes_recovered': 6586368, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7669}, {'shards': '0,2', 'objects': 1990}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:50.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10477", 'reported_seq': 10606, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.667868+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:45.667868+0000', 'last_peered': '2024-09-11T05:17:45.667868+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:45.667868+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39833600, 'num_objects': 9725, 'num_object_clones': 0, 'num_object_copies': 19450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9725, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7642, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6149}, {'shards': '0,2', 'objects': 3203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10362", 'reported_seq': 10493, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:46.306563+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:46.306563+0000', 'last_peered': '2024-09-11T05:17:46.306563+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:46.306563+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39358464, 'num_objects': 9609, 'num_object_clones': 0, 'num_object_copies': 19218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9609, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8430, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3152}, {'shards': '2', 'objects': 6084}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10190", 'reported_seq': 10355, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.730106+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:45.730106+0000', 'last_peered': '2024-09-11T05:17:45.730106+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:45.730106+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38838272, 'num_objects': 9482, 'num_object_clones': 0, 'num_object_copies': 18964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6943, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6040}, {'shards': '0,2', 'objects': 3092}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10356", 'reported_seq': 10474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:46.295116+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:46.295116+0000', 'last_peered': '2024-09-11T05:17:46.295116+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:46.295116+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39636992, 'num_objects': 9677, 'num_object_clones': 0, 'num_object_copies': 19354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9677, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8385, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3185}, {'shards': '2', 'objects': 6155}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10428", 'reported_seq': 10560, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.754109+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:45.754109+0000', 'last_peered': '2024-09-11T05:17:45.754109+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:45.754109+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39796736, 'num_objects': 9716, 'num_object_clones': 0, 'num_object_copies': 19432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7461, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6093}, {'shards': '0,2', 'objects': 3268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:50.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10162", 'reported_seq': 10293, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.717467+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:45.717467+0000', 'last_peered': '2024-09-11T05:17:45.717467+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:45.717467+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38879232, 'num_objects': 9492, 'num_object_clones': 0, 'num_object_copies': 18984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7028, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5997}, {'shards': '0,2', 'objects': 3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.230 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:50.230 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10255", 'reported_seq': 10390, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.725539+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:45.725539+0000', 'last_peered': '2024-09-11T05:17:45.725539+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:45.725539+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39096320, 'num_objects': 9545, 'num_object_clones': 0, 'num_object_copies': 19090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7013, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6001}, {'shards': '0,2', 'objects': 3190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.230 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:50.230 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10147", 'reported_seq': 10291, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:45.766740+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:45.766740+0000', 'last_peered': '2024-09-11T05:17:45.766740+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:45.766740+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38679145, 'num_objects': 9444, 'num_object_clones': 0, 'num_object_copies': 18888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7154, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5929}, {'shards': '0,2', 'objects': 3169}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:50.230 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:51.257 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:51.257 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/307212 objects degraded (11.533%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":349,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153606,"data_bytes":629154452,"bytes_used":2168209408,"bytes_avail":287742083072,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":307212,"degraded_ratio":0.11533078134968684,"misplaced_objects":1990,"misplaced_total":307212,"misplaced_ratio":0.0064776115516320979,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":54588,"recovering_keys_per_sec":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":371},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:51.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:51.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:52.181 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:52.236 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-11T05:17:51.489655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620020372,"num_objects":151376,"num_object_clones":0,"num_object_copies":302752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":151376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130323,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":21217299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2097788,"kb_used_data":1254476,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":281017732,"statfs":{"total":289910292480,"available":287762157568,"internally_reserved":0,"allocated":1284583424,"data_stored":1283732984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":863489025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,9,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27389952,"num_objects":-6687,"num_object_clones":0,"num_object_copies":-13374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-253,"num_objects_unfound":0,"num_objects_dirty":-6687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6687,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":1036288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005369"},"pg_stats":[{"pgid":"2.f","version":"45'10586","reported_seq":10665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.709975+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:50.709975+0000","last_peered":"2024-09-11T05:17:50.709975+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:50.709975+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10586,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5929},{"shards":"0,2","objects":3183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10465","reported_seq":10568,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.317928+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:51.317928+0000","last_peered":"2024-09-11T05:17:51.317928+0000","last_clean":"2024-09-11T05:17:51.317928+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:51.317928+0000","last_undegraded":"2024-09-11T05:17:51.317928+0000","last_fullsized":"2024-09-11T05:17:51.317928+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8863,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10356","reported_seq":10481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.359332+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:51.359332+0000","last_peered":"2024-09-11T05:17:51.359332+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:51.359332+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342680,"num_objects":9362,"num_object_clones":0,"num_object_copies":18724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8759,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2999},{"shards":"2","objects":5868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10472","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.226996+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:51.226996+0000","last_peered":"2024-09-11T05:17:51.226996+0000","last_clean":"2024-09-11T05:17:51.226996+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:51.226996+0000","last_undegraded":"2024-09-11T05:17:51.226996+0000","last_fullsized":"2024-09-11T05:17:51.226996+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":9488,"num_object_clones":0,"num_object_copies":18976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10472,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10564","reported_seq":10693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:50.643165+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:50.643165+0000","last_peered":"2024-09-11T05:17:50.643165+0000","last_clean":"2024-09-11T05:17:50.643165+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:50.643165+0000","last_undegraded":"2024-09-11T05:17:50.643165+0000","last_fullsized":"2024-09-11T05:17:50.643165+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39342080,"num_objects":9605,"num_object_clones":0,"num_object_copies":19210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7398,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10507","reported_seq":14059,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.821015+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:50.821015+0000","last_peered":"2024-09-11T05:17:50.821015+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:50.821015+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7629},{"shards":"0,2","objects":1897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10633","reported_seq":10761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.797592+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:50.797592+0000","last_peered":"2024-09-11T05:17:50.797592+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:50.797592+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7798,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5993},{"shards":"0,2","objects":3047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10465","reported_seq":10589,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:50.769217+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:50.769217+0000","last_peered":"2024-09-11T05:17:50.769217+0000","last_clean":"2024-09-11T05:17:50.769217+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:50.769217+0000","last_undegraded":"2024-09-11T05:17:50.769217+0000","last_fullsized":"2024-09-11T05:17:50.769217+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38580224,"num_objects":9419,"num_object_clones":0,"num_object_copies":18838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10489","reported_seq":10621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.340097+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:51.340097+0000","last_peered":"2024-09-11T05:17:51.340097+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:51.340097+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38838272,"num_objects":9482,"num_object_clones":0,"num_object_copies":18964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3025},{"shards":"2","objects":5957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10316","reported_seq":10481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.644269+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:50.644269+0000","last_peered":"2024-09-11T05:17:50.644269+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:50.644269+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38322176,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10482","reported_seq":10596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.212172+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:51.212172+0000","last_peered":"2024-09-11T05:17:51.212172+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:51.212172+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39120896,"num_objects":9551,"num_object_clones":0,"num_object_copies":19102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8511,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3059},{"shards":"2","objects":6029}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10337","reported_seq":17423,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.333579+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:51.333579+0000","last_peered":"2024-09-11T05:17:51.333579+0000","last_clean":"2024-09-11T05:17:51.333579+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:51.333579+0000","last_undegraded":"2024-09-11T05:17:51.333579+0000","last_fullsized":"2024-09-11T05:17:51.333579+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38383616,"num_objects":9371,"num_object_clones":0,"num_object_copies":18742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8240,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10567","reported_seq":10699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.780633+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:50.780633+0000","last_peered":"2024-09-11T05:17:50.780633+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:50.780633+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5954},{"shards":"0,2","objects":3129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10312","reported_seq":10446,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.766074+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:50.766074+0000","last_peered":"2024-09-11T05:17:50.766074+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:50.766074+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7178,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5847},{"shards":"0,2","objects":3014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10394","reported_seq":10529,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.733758+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:50.733758+0000","last_peered":"2024-09-11T05:17:50.733758+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:50.733758+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7152,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10312","reported_seq":10456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.674520+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:50.674520+0000","last_peered":"2024-09-11T05:17:50.674520+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:50.674520+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38003305,"num_objects":9279,"num_object_clones":0,"num_object_copies":18558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7319,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5764},{"shards":"0,2","objects":3004}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620020353,"num_objects":151374,"num_object_clones":0,"num_object_copies":302748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":151374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130321,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5182,"num_bytes_recovered":21217280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281994752,"data_stored":1281906591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670092,"kb_used_data":417660,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93701748,"statfs":{"total":96636764160,"available":95950589952,"internally_reserved":0,"allocated":427683840,"data_stored":427424914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577740,"kb_used_data":325308,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93794100,"statfs":{"total":96636764160,"available":96045158400,"internally_reserved":0,"allocated":333115392,"data_stored":332782809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215168,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849956,"kb_used_data":511508,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93521884,"statfs":{"total":96636764160,"available":95766409216,"internally_reserved":0,"allocated":523784192,"data_stored":523525261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":522792960,"data_stored":522789481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332603392,"data_stored":332526261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426598400,"data_stored":426590849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:55.238 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:56.054 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:56.055 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:56.167 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-11T05:17:55.490745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620020372,"num_objects":151376,"num_object_clones":0,"num_object_copies":302752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":151376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130323,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":21217299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2119244,"kb_used_data":1253724,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280996276,"statfs":{"total":289910292480,"available":287740186624,"internally_reserved":0,"allocated":1283813376,"data_stored":1282969894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":886230017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,5,12,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18370560,"num_objects":-4485,"num_object_clones":0,"num_object_copies":-8970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-168,"num_objects_unfound":0,"num_objects_dirty":-4485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4485,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005467"},"pg_stats":[{"pgid":"2.f","version":"45'10586","reported_seq":10665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.709975+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:50.709975+0000","last_peered":"2024-09-11T05:17:50.709975+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:50.709975+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10586,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5929},{"shards":"0,2","objects":3183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10465","reported_seq":10568,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.317928+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:51.317928+0000","last_peered":"2024-09-11T05:17:51.317928+0000","last_clean":"2024-09-11T05:17:51.317928+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:51.317928+0000","last_undegraded":"2024-09-11T05:17:51.317928+0000","last_fullsized":"2024-09-11T05:17:51.317928+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8863,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10356","reported_seq":10481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.359332+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:51.359332+0000","last_peered":"2024-09-11T05:17:51.359332+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:51.359332+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342680,"num_objects":9362,"num_object_clones":0,"num_object_copies":18724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8759,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2999},{"shards":"2","objects":5868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10472","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.226996+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:51.226996+0000","last_peered":"2024-09-11T05:17:51.226996+0000","last_clean":"2024-09-11T05:17:51.226996+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:51.226996+0000","last_undegraded":"2024-09-11T05:17:51.226996+0000","last_fullsized":"2024-09-11T05:17:51.226996+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":9488,"num_object_clones":0,"num_object_copies":18976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10472,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10564","reported_seq":10693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:50.643165+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:50.643165+0000","last_peered":"2024-09-11T05:17:50.643165+0000","last_clean":"2024-09-11T05:17:50.643165+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:50.643165+0000","last_undegraded":"2024-09-11T05:17:50.643165+0000","last_fullsized":"2024-09-11T05:17:50.643165+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39342080,"num_objects":9605,"num_object_clones":0,"num_object_copies":19210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7398,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10507","reported_seq":14059,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.821015+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:50.821015+0000","last_peered":"2024-09-11T05:17:50.821015+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:50.821015+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7629},{"shards":"0,2","objects":1897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10633","reported_seq":10761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.797592+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:50.797592+0000","last_peered":"2024-09-11T05:17:50.797592+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:50.797592+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7798,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5993},{"shards":"0,2","objects":3047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10465","reported_seq":10589,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:50.769217+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:50.769217+0000","last_peered":"2024-09-11T05:17:50.769217+0000","last_clean":"2024-09-11T05:17:50.769217+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:50.769217+0000","last_undegraded":"2024-09-11T05:17:50.769217+0000","last_fullsized":"2024-09-11T05:17:50.769217+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38580224,"num_objects":9419,"num_object_clones":0,"num_object_copies":18838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10489","reported_seq":10621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.340097+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:51.340097+0000","last_peered":"2024-09-11T05:17:51.340097+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:51.340097+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38838272,"num_objects":9482,"num_object_clones":0,"num_object_copies":18964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3025},{"shards":"2","objects":5957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10316","reported_seq":10481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.644269+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:50.644269+0000","last_peered":"2024-09-11T05:17:50.644269+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:50.644269+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38322176,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10482","reported_seq":10596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.212172+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:51.212172+0000","last_peered":"2024-09-11T05:17:51.212172+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:51.212172+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39120896,"num_objects":9551,"num_object_clones":0,"num_object_copies":19102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8511,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3059},{"shards":"2","objects":6029}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10337","reported_seq":17423,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.333579+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:51.333579+0000","last_peered":"2024-09-11T05:17:51.333579+0000","last_clean":"2024-09-11T05:17:51.333579+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:51.333579+0000","last_undegraded":"2024-09-11T05:17:51.333579+0000","last_fullsized":"2024-09-11T05:17:51.333579+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38383616,"num_objects":9371,"num_object_clones":0,"num_object_copies":18742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8240,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10567","reported_seq":10699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.780633+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:50.780633+0000","last_peered":"2024-09-11T05:17:50.780633+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:50.780633+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5954},{"shards":"0,2","objects":3129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10312","reported_seq":10446,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.766074+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:50.766074+0000","last_peered":"2024-09-11T05:17:50.766074+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:50.766074+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7178,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5847},{"shards":"0,2","objects":3014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10394","reported_seq":10529,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.733758+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:50.733758+0000","last_peered":"2024-09-11T05:17:50.733758+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:50.733758+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7152,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10312","reported_seq":10456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.674520+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:50.674520+0000","last_peered":"2024-09-11T05:17:50.674520+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:50.674520+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38003305,"num_objects":9279,"num_object_clones":0,"num_object_copies":18558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7319,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5764},{"shards":"0,2","objects":3004}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620020353,"num_objects":151374,"num_object_clones":0,"num_object_copies":302748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":151374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130321,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5182,"num_bytes_recovered":21217280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281261568,"data_stored":1281180365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670092,"kb_used_data":417660,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93701748,"statfs":{"total":96636764160,"available":95950589952,"internally_reserved":0,"allocated":427683840,"data_stored":427424914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599196,"kb_used_data":324556,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93772644,"statfs":{"total":96636764160,"available":96023187456,"internally_reserved":0,"allocated":332345344,"data_stored":332019719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215168,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849956,"kb_used_data":511508,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93521884,"statfs":{"total":96636764160,"available":95766409216,"internally_reserved":0,"allocated":523784192,"data_stored":523525261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":522792960,"data_stored":522789481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331870208,"data_stored":331800035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426598400,"data_stored":426590849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:56.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:57.048 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:57.050 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:57.103 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-11T05:17:55.490745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620020372,"num_objects":151376,"num_object_clones":0,"num_object_copies":302752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":151376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130323,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":21217299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2119244,"kb_used_data":1253724,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280996276,"statfs":{"total":289910292480,"available":287740186624,"internally_reserved":0,"allocated":1283813376,"data_stored":1282969894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":886230017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,5,12,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18370560,"num_objects":-4485,"num_object_clones":0,"num_object_copies":-8970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-168,"num_objects_unfound":0,"num_objects_dirty":-4485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4485,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005467"},"pg_stats":[{"pgid":"2.f","version":"45'10586","reported_seq":10665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.709975+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:50.709975+0000","last_peered":"2024-09-11T05:17:50.709975+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:50.709975+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10586,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5929},{"shards":"0,2","objects":3183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10465","reported_seq":10568,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.317928+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:51.317928+0000","last_peered":"2024-09-11T05:17:51.317928+0000","last_clean":"2024-09-11T05:17:51.317928+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:51.317928+0000","last_undegraded":"2024-09-11T05:17:51.317928+0000","last_fullsized":"2024-09-11T05:17:51.317928+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8863,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10356","reported_seq":10481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.359332+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:51.359332+0000","last_peered":"2024-09-11T05:17:51.359332+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:51.359332+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342680,"num_objects":9362,"num_object_clones":0,"num_object_copies":18724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8759,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2999},{"shards":"2","objects":5868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10472","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.226996+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:51.226996+0000","last_peered":"2024-09-11T05:17:51.226996+0000","last_clean":"2024-09-11T05:17:51.226996+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:51.226996+0000","last_undegraded":"2024-09-11T05:17:51.226996+0000","last_fullsized":"2024-09-11T05:17:51.226996+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":9488,"num_object_clones":0,"num_object_copies":18976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10472,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10564","reported_seq":10693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:50.643165+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:50.643165+0000","last_peered":"2024-09-11T05:17:50.643165+0000","last_clean":"2024-09-11T05:17:50.643165+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:50.643165+0000","last_undegraded":"2024-09-11T05:17:50.643165+0000","last_fullsized":"2024-09-11T05:17:50.643165+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39342080,"num_objects":9605,"num_object_clones":0,"num_object_copies":19210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7398,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10507","reported_seq":14059,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.821015+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:50.821015+0000","last_peered":"2024-09-11T05:17:50.821015+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:50.821015+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7629},{"shards":"0,2","objects":1897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10633","reported_seq":10761,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.797592+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:50.797592+0000","last_peered":"2024-09-11T05:17:50.797592+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:50.797592+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7798,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5993},{"shards":"0,2","objects":3047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10465","reported_seq":10589,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:50.769217+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:50.769217+0000","last_peered":"2024-09-11T05:17:50.769217+0000","last_clean":"2024-09-11T05:17:50.769217+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:50.769217+0000","last_undegraded":"2024-09-11T05:17:50.769217+0000","last_fullsized":"2024-09-11T05:17:50.769217+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38580224,"num_objects":9419,"num_object_clones":0,"num_object_copies":18838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10489","reported_seq":10621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.340097+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:51.340097+0000","last_peered":"2024-09-11T05:17:51.340097+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:51.340097+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38838272,"num_objects":9482,"num_object_clones":0,"num_object_copies":18964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3025},{"shards":"2","objects":5957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10316","reported_seq":10481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.644269+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:50.644269+0000","last_peered":"2024-09-11T05:17:50.644269+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:50.644269+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38322176,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10482","reported_seq":10596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:51.212172+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:51.212172+0000","last_peered":"2024-09-11T05:17:51.212172+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:51.212172+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39120896,"num_objects":9551,"num_object_clones":0,"num_object_copies":19102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8511,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3059},{"shards":"2","objects":6029}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10337","reported_seq":17423,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:51.333579+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:51.333579+0000","last_peered":"2024-09-11T05:17:51.333579+0000","last_clean":"2024-09-11T05:17:51.333579+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:51.333579+0000","last_undegraded":"2024-09-11T05:17:51.333579+0000","last_fullsized":"2024-09-11T05:17:51.333579+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38383616,"num_objects":9371,"num_object_clones":0,"num_object_copies":18742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8240,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10567","reported_seq":10699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.780633+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:50.780633+0000","last_peered":"2024-09-11T05:17:50.780633+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:50.780633+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5954},{"shards":"0,2","objects":3129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10312","reported_seq":10446,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.766074+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:50.766074+0000","last_peered":"2024-09-11T05:17:50.766074+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:50.766074+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7178,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5847},{"shards":"0,2","objects":3014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10394","reported_seq":10529,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.733758+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:50.733758+0000","last_peered":"2024-09-11T05:17:50.733758+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:50.733758+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7152,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10312","reported_seq":10456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:50.674520+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:50.674520+0000","last_peered":"2024-09-11T05:17:50.674520+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:50.674520+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38003305,"num_objects":9279,"num_object_clones":0,"num_object_copies":18558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7319,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5764},{"shards":"0,2","objects":3004}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620020353,"num_objects":151374,"num_object_clones":0,"num_object_copies":302748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1897,"num_objects_unfound":0,"num_objects_dirty":151374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130321,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5182,"num_bytes_recovered":21217280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281261568,"data_stored":1281180365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670092,"kb_used_data":417660,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93701748,"statfs":{"total":96636764160,"available":95950589952,"internally_reserved":0,"allocated":427683840,"data_stored":427424914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599196,"kb_used_data":324556,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93772644,"statfs":{"total":96636764160,"available":96023187456,"internally_reserved":0,"allocated":332345344,"data_stored":332019719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215168,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849956,"kb_used_data":511508,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93521884,"statfs":{"total":96636764160,"available":95766409216,"internally_reserved":0,"allocated":523784192,"data_stored":523525261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":522792960,"data_stored":522789481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331870208,"data_stored":331800035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426598400,"data_stored":426590849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:17:57.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10586", 'reported_seq': 10665, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.709975+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:17:50.709975+0000', 'last_peered': '2024-09-11T05:17:50.709975+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:17:50.709975+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39337984, 'num_objects': 9604, 'num_object_clones': 0, 'num_object_copies': 19208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10586, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5929}, {'shards': '0,2', 'objects': 3183}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10356", 'reported_seq': 10481, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:51.359332+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:17:51.359332+0000', 'last_peered': '2024-09-11T05:17:51.359332+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:17:51.359332+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38342680, 'num_objects': 9362, 'num_object_clones': 0, 'num_object_copies': 18724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9362, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8759, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2999}, {'shards': '2', 'objects': 5868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10507", 'reported_seq': 14059, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.821015+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:17:50.821015+0000', 'last_peered': '2024-09-11T05:17:50.821015+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:17:50.821015+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39026688, 'num_objects': 9528, 'num_object_clones': 0, 'num_object_copies': 19056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1897, 'num_objects_unfound': 0, 'num_objects_dirty': 9528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7051, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1701, 'num_bytes_recovered': 6967296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7629}, {'shards': '0,2', 'objects': 1897}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10633", 'reported_seq': 10761, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.797592+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:17:50.797592+0000', 'last_peered': '2024-09-11T05:17:50.797592+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:17:50.797592+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39194624, 'num_objects': 9569, 'num_object_clones': 0, 'num_object_copies': 19138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7798, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5993}, {'shards': '0,2', 'objects': 3047}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:17:57.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10489", 'reported_seq': 10621, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:51.340097+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:17:51.340097+0000', 'last_peered': '2024-09-11T05:17:51.340097+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:17:51.340097+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38838272, 'num_objects': 9482, 'num_object_clones': 0, 'num_object_copies': 18964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8557, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3025}, {'shards': '2', 'objects': 5957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10316", 'reported_seq': 10481, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.644269+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:17:50.644269+0000', 'last_peered': '2024-09-11T05:17:50.644269+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:17:50.644269+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38322176, 'num_objects': 9356, 'num_object_clones': 0, 'num_object_copies': 18712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7069, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5914}, {'shards': '0,2', 'objects': 2966}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10482", 'reported_seq': 10596, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:51.212172+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:17:51.212172+0000', 'last_peered': '2024-09-11T05:17:51.212172+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:17:51.212172+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39120896, 'num_objects': 9551, 'num_object_clones': 0, 'num_object_copies': 19102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8511, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3059}, {'shards': '2', 'objects': 6029}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10567", 'reported_seq': 10699, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.780633+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:17:50.780633+0000', 'last_peered': '2024-09-11T05:17:50.780633+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:17:50.780633+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39227392, 'num_objects': 9577, 'num_object_clones': 0, 'num_object_copies': 19154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7600, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5954}, {'shards': '0,2', 'objects': 3129}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10312", 'reported_seq': 10446, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.766074+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:17:50.766074+0000', 'last_peered': '2024-09-11T05:17:50.766074+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:17:50.766074+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38264832, 'num_objects': 9342, 'num_object_clones': 0, 'num_object_copies': 18684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7178, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5847}, {'shards': '0,2', 'objects': 3014}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:17:57.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10394", 'reported_seq': 10529, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.733758+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:17:50.733758+0000', 'last_peered': '2024-09-11T05:17:50.733758+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:17:50.733758+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38526976, 'num_objects': 9406, 'num_object_clones': 0, 'num_object_copies': 18812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7152, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5862}, {'shards': '0,2', 'objects': 3051}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:17:57.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10312", 'reported_seq': 10456, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:17:50.674520+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:17:50.674520+0000', 'last_peered': '2024-09-11T05:17:50.674520+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:17:50.674520+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38003305, 'num_objects': 9279, 'num_object_clones': 0, 'num_object_copies': 18558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7319, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5764}, {'shards': '0,2', 'objects': 3004}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:17:57.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:17:58.083 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:58.083 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/298252 objects degraded (11.880%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":355,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149126,"data_bytes":610804372,"bytes_used":2167234560,"bytes_avail":287743057920,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":298252,"degraded_ratio":0.1187955152019098,"misplaced_objects":1803,"misplaced_total":298252,"misplaced_ratio":0.0060452235022732444,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":89407,"recovering_keys_per_sec":0,"num_objects_recovered":264,"num_bytes_recovered":1073152,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":561},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:17:58.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:17:58.170 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:17:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:17:58.911 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:17:58.965 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-11T05:17:57.491174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":610804372,"num_objects":149126,"num_object_clones":0,"num_object_copies":298252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1803,"num_objects_unfound":0,"num_objects_dirty":149126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132573,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5279,"num_bytes_recovered":21602323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2116440,"kb_used_data":1242588,"kb_used_omap":12,"kb_used_meta":873779,"kb_avail":280999080,"statfs":{"total":289910292480,"available":287743057920,"internally_reserved":0,"allocated":1272410112,"data_stored":1271566630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":894749697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,10,10,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27586560,"num_objects":-6735,"num_object_clones":0,"num_object_copies":-13470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":-6735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6735,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002890"},"pg_stats":[{"pgid":"2.f","version":"45'10735","reported_seq":10814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.875480+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:17:55.875480+0000","last_peered":"2024-09-11T05:17:55.875480+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:17:55.875480+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38727680,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10735,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5780},{"shards":"0,2","objects":3034}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10607","reported_seq":10710,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:56.362366+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:17:56.362366+0000","last_peered":"2024-09-11T05:17:56.362366+0000","last_clean":"2024-09-11T05:17:56.362366+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:17:56.362366+0000","last_undegraded":"2024-09-11T05:17:56.362366+0000","last_fullsized":"2024-09-11T05:17:56.362366+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38064128,"num_objects":9293,"num_object_clones":0,"num_object_copies":18586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10504","reported_seq":10630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:56.331947+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:17:56.331947+0000","last_peered":"2024-09-11T05:17:56.331947+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:17:56.331947+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37736472,"num_objects":9214,"num_object_clones":0,"num_object_copies":18428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8907,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2851},{"shards":"2","objects":5720}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10621","reported_seq":10693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:56.397415+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:17:56.397415+0000","last_peered":"2024-09-11T05:17:56.397415+0000","last_clean":"2024-09-11T05:17:56.397415+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:17:56.397415+0000","last_undegraded":"2024-09-11T05:17:56.397415+0000","last_fullsized":"2024-09-11T05:17:56.397415+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10621,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10694","reported_seq":10824,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:55.855380+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:17:55.855380+0000","last_peered":"2024-09-11T05:17:55.855380+0000","last_clean":"2024-09-11T05:17:55.855380+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:17:55.855380+0000","last_undegraded":"2024-09-11T05:17:55.855380+0000","last_fullsized":"2024-09-11T05:17:55.855380+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38809600,"num_objects":9475,"num_object_clones":0,"num_object_copies":18950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7528,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10644","reported_seq":14386,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.893446+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:17:55.893446+0000","last_peered":"2024-09-11T05:17:55.893446+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:17:55.893446+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38465536,"num_objects":9391,"num_object_clones":0,"num_object_copies":18782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1803,"num_objects_unfound":0,"num_objects_dirty":9391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7188,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1797,"num_bytes_recovered":7352320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7586},{"shards":"0,2","objects":1803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10780","reported_seq":10913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.866865+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:17:55.866865+0000","last_peered":"2024-09-11T05:17:55.866865+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:17:55.866865+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38592512,"num_objects":9422,"num_object_clones":0,"num_object_copies":18844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7945,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":2900}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10600","reported_seq":10724,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:55.784461+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:17:55.784461+0000","last_peered":"2024-09-11T05:17:55.784461+0000","last_clean":"2024-09-11T05:17:55.784461+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:17:55.784461+0000","last_undegraded":"2024-09-11T05:17:55.784461+0000","last_fullsized":"2024-09-11T05:17:55.784461+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38027264,"num_objects":9284,"num_object_clones":0,"num_object_copies":18568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10614","reported_seq":10746,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:56.310666+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:17:56.310666+0000","last_peered":"2024-09-11T05:17:56.310666+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:17:56.310666+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8682,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2900},{"shards":"2","objects":5832}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10438","reported_seq":10603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.648870+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:17:55.648870+0000","last_peered":"2024-09-11T05:17:55.648870+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:17:55.648870+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37822464,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7191,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5792},{"shards":"0,2","objects":2844}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10609","reported_seq":10727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:56.394362+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:17:56.394362+0000","last_peered":"2024-09-11T05:17:56.394362+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:17:56.394362+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38600704,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8638,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2932},{"shards":"2","objects":5902}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10486","reported_seq":17572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:56.329031+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:17:56.329031+0000","last_peered":"2024-09-11T05:17:56.329031+0000","last_clean":"2024-09-11T05:17:56.329031+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:17:56.329031+0000","last_undegraded":"2024-09-11T05:17:56.329031+0000","last_fullsized":"2024-09-11T05:17:56.329031+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37773312,"num_objects":9222,"num_object_clones":0,"num_object_copies":18444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10707","reported_seq":10839,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.827163+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:17:55.827163+0000","last_peered":"2024-09-11T05:17:55.827163+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:17:55.827163+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38653952,"num_objects":9437,"num_object_clones":0,"num_object_copies":18874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5814},{"shards":"0,2","objects":2989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10457","reported_seq":10591,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.628065+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:17:55.628065+0000","last_peered":"2024-09-11T05:17:55.628065+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:17:55.628065+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37670912,"num_objects":9197,"num_object_clones":0,"num_object_copies":18394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7323,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":2869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10548","reported_seq":10683,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.786857+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:17:55.786857+0000","last_peered":"2024-09-11T05:17:55.786857+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:17:55.786857+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37896192,"num_objects":9252,"num_object_clones":0,"num_object_copies":18504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":2897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10463","reported_seq":10607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:17:55.841881+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:17:55.841881+0000","last_peered":"2024-09-11T05:17:55.841881+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:17:55.841881+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384809,"num_objects":9128,"num_object_clones":0,"num_object_copies":18256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7470,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":610804353,"num_objects":149124,"num_object_clones":0,"num_object_copies":298248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1803,"num_objects_unfound":0,"num_objects_dirty":149124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132571,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5278,"num_bytes_recovered":21602304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1269817344,"data_stored":1269736141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160707,"ondisk_log_size":160707,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669864,"kb_used_data":413260,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93701976,"statfs":{"total":96636764160,"available":95950823424,"internally_reserved":0,"allocated":423178240,"data_stored":422919314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599196,"kb_used_data":324556,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93772644,"statfs":{"total":96636764160,"available":96023187456,"internally_reserved":0,"allocated":332345344,"data_stored":332019719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215169,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847380,"kb_used_data":504772,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93524460,"statfs":{"total":96636764160,"available":95769047040,"internally_reserved":0,"allocated":516886528,"data_stored":516627597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":515874816,"data_stored":515871337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331870208,"data_stored":331800035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422072320,"data_stored":422064769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:01.968 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:02.775 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:02.777 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:02.820 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-11T05:18:01.496591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601723540,"num_objects":146909,"num_object_clones":0,"num_object_copies":293818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":146909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134790,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5359,"num_bytes_recovered":21930003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2095744,"kb_used_data":1230220,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":281019776,"statfs":{"total":289910292480,"available":287764250624,"internally_reserved":0,"allocated":1259745280,"data_stored":1258901798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":886230017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,8,6,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27430912,"num_objects":-6697,"num_object_clones":0,"num_object_copies":-13394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-267,"num_objects_unfound":0,"num_objects_dirty":-6697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6697,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":1093632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007423"},"pg_stats":[{"pgid":"2.f","version":"45'10871","reported_seq":10948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.949603+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:00.949603+0000","last_peered":"2024-09-11T05:18:00.949603+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:00.949603+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38170624,"num_objects":9319,"num_object_clones":0,"num_object_copies":18638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10871,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5644},{"shards":"0,2","objects":2898}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10737","reported_seq":10840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.378258+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:01.378258+0000","last_peered":"2024-09-11T05:18:01.378258+0000","last_clean":"2024-09-11T05:18:01.378258+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:01.378258+0000","last_undegraded":"2024-09-11T05:18:01.378258+0000","last_fullsized":"2024-09-11T05:18:01.378258+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37531648,"num_objects":9163,"num_object_clones":0,"num_object_copies":18326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9135,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10639","reported_seq":10765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.401955+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:01.401955+0000","last_peered":"2024-09-11T05:18:01.401955+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:01.401955+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37183512,"num_objects":9079,"num_object_clones":0,"num_object_copies":18158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9042,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2716},{"shards":"2","objects":5585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10755","reported_seq":10827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.395401+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:01.395401+0000","last_peered":"2024-09-11T05:18:01.395401+0000","last_clean":"2024-09-11T05:18:01.395401+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:01.395401+0000","last_undegraded":"2024-09-11T05:18:01.395401+0000","last_fullsized":"2024-09-11T05:18:01.395401+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37703680,"num_objects":9205,"num_object_clones":0,"num_object_copies":18410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10755,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10831","reported_seq":10961,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:00.898329+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:00.898329+0000","last_peered":"2024-09-11T05:18:00.898329+0000","last_clean":"2024-09-11T05:18:00.898329+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:00.898329+0000","last_undegraded":"2024-09-11T05:18:00.898329+0000","last_fullsized":"2024-09-11T05:18:00.898329+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10776","reported_seq":14679,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.922993+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:00.922993+0000","last_peered":"2024-09-11T05:18:00.922993+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:00.922993+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7320,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":7680000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7534},{"shards":"0,2","objects":1723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10912","reported_seq":11045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.876270+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:00.876270+0000","last_peered":"2024-09-11T05:18:00.876270+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:00.876270+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8077,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10746","reported_seq":10870,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:00.926732+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:00.926732+0000","last_peered":"2024-09-11T05:18:00.926732+0000","last_clean":"2024-09-11T05:18:00.926732+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:00.926732+0000","last_undegraded":"2024-09-11T05:18:00.926732+0000","last_fullsized":"2024-09-11T05:18:00.926732+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8049,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10767","reported_seq":10899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.396173+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:01.396173+0000","last_peered":"2024-09-11T05:18:01.396173+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:01.396173+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8835,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2747},{"shards":"2","objects":5679}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10572","reported_seq":10737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.754021+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:00.754021+0000","last_peered":"2024-09-11T05:18:00.754021+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:00.754021+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37273600,"num_objects":9100,"num_object_clones":0,"num_object_copies":18200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":2710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10754","reported_seq":10872,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.422589+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:01.422589+0000","last_peered":"2024-09-11T05:18:01.422589+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:01.422589+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38006784,"num_objects":9279,"num_object_clones":0,"num_object_copies":18558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8783,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2787},{"shards":"2","objects":5757}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10629","reported_seq":17715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.423377+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:01.423377+0000","last_peered":"2024-09-11T05:18:01.423377+0000","last_clean":"2024-09-11T05:18:01.423377+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:01.423377+0000","last_undegraded":"2024-09-11T05:18:01.423377+0000","last_fullsized":"2024-09-11T05:18:01.423377+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37187584,"num_objects":9079,"num_object_clones":0,"num_object_copies":18158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8532,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10854","reported_seq":10986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.899376+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:00.899376+0000","last_peered":"2024-09-11T05:18:00.899376+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:00.899376+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7887,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":2842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10606","reported_seq":10740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.926976+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:00.926976+0000","last_peered":"2024-09-11T05:18:00.926976+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:00.926976+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37060608,"num_objects":9048,"num_object_clones":0,"num_object_copies":18096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7472,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5553},{"shards":"0,2","objects":2720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10677","reported_seq":10812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.928449+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:00.928449+0000","last_peered":"2024-09-11T05:18:00.928449+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:00.928449+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7435,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10598","reported_seq":10742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.856717+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:00.856717+0000","last_peered":"2024-09-11T05:18:00.856717+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:00.856717+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36831849,"num_objects":8993,"num_object_clones":0,"num_object_copies":17986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7605,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5478},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":601723521,"num_objects":146907,"num_object_clones":0,"num_object_copies":293814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":146907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134788,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5358,"num_bytes_recovered":21929984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1257623552,"data_stored":1257542349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656328,"kb_used_data":408056,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93715512,"statfs":{"total":96636764160,"available":95964684288,"internally_reserved":0,"allocated":417849344,"data_stored":417590418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590100,"kb_used_data":323780,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93781740,"statfs":{"total":96636764160,"available":96032501760,"internally_reserved":0,"allocated":331550720,"data_stored":331225095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,3,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215170,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849316,"kb_used_data":498384,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93522524,"statfs":{"total":96636764160,"available":95767064576,"internally_reserved":0,"allocated":510345216,"data_stored":510086285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359329451},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509161472,"data_stored":509157993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331038720,"data_stored":330968547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417423360,"data_stored":417415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:02.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:03.627 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:03.630 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-11T05:18:03.497090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601723540,"num_objects":146909,"num_object_clones":0,"num_object_copies":293818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":146909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134790,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5359,"num_bytes_recovered":21930003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2099136,"kb_used_data":1229452,"kb_used_omap":12,"kb_used_meta":869619,"kb_avail":281016384,"statfs":{"total":289910292480,"available":287760777216,"internally_reserved":0,"allocated":1258958848,"data_stored":1258115366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":890489857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,11,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18296832,"num_objects":-4467,"num_object_clones":0,"num_object_copies":-8934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-174,"num_objects_unfound":0,"num_objects_dirty":-4467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4467,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":712704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007434"},"pg_stats":[{"pgid":"2.f","version":"45'10871","reported_seq":10948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.949603+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:00.949603+0000","last_peered":"2024-09-11T05:18:00.949603+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:00.949603+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38170624,"num_objects":9319,"num_object_clones":0,"num_object_copies":18638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10871,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5644},{"shards":"0,2","objects":2898}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10737","reported_seq":10840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.378258+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:01.378258+0000","last_peered":"2024-09-11T05:18:01.378258+0000","last_clean":"2024-09-11T05:18:01.378258+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:01.378258+0000","last_undegraded":"2024-09-11T05:18:01.378258+0000","last_fullsized":"2024-09-11T05:18:01.378258+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37531648,"num_objects":9163,"num_object_clones":0,"num_object_copies":18326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9135,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10639","reported_seq":10765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.401955+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:01.401955+0000","last_peered":"2024-09-11T05:18:01.401955+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:01.401955+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37183512,"num_objects":9079,"num_object_clones":0,"num_object_copies":18158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9042,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2716},{"shards":"2","objects":5585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10755","reported_seq":10827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.395401+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:01.395401+0000","last_peered":"2024-09-11T05:18:01.395401+0000","last_clean":"2024-09-11T05:18:01.395401+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:01.395401+0000","last_undegraded":"2024-09-11T05:18:01.395401+0000","last_fullsized":"2024-09-11T05:18:01.395401+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37703680,"num_objects":9205,"num_object_clones":0,"num_object_copies":18410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10755,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10831","reported_seq":10961,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:00.898329+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:00.898329+0000","last_peered":"2024-09-11T05:18:00.898329+0000","last_clean":"2024-09-11T05:18:00.898329+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:00.898329+0000","last_undegraded":"2024-09-11T05:18:00.898329+0000","last_fullsized":"2024-09-11T05:18:00.898329+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10776","reported_seq":14679,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.922993+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:00.922993+0000","last_peered":"2024-09-11T05:18:00.922993+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:00.922993+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7320,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":7680000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7534},{"shards":"0,2","objects":1723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10912","reported_seq":11045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.876270+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:00.876270+0000","last_peered":"2024-09-11T05:18:00.876270+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:00.876270+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8077,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10746","reported_seq":10870,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:00.926732+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:00.926732+0000","last_peered":"2024-09-11T05:18:00.926732+0000","last_clean":"2024-09-11T05:18:00.926732+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:00.926732+0000","last_undegraded":"2024-09-11T05:18:00.926732+0000","last_fullsized":"2024-09-11T05:18:00.926732+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8049,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10767","reported_seq":10899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.396173+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:01.396173+0000","last_peered":"2024-09-11T05:18:01.396173+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:01.396173+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8835,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2747},{"shards":"2","objects":5679}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10572","reported_seq":10737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.754021+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:00.754021+0000","last_peered":"2024-09-11T05:18:00.754021+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:00.754021+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37273600,"num_objects":9100,"num_object_clones":0,"num_object_copies":18200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":2710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10754","reported_seq":10872,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.422589+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:01.422589+0000","last_peered":"2024-09-11T05:18:01.422589+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:01.422589+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38006784,"num_objects":9279,"num_object_clones":0,"num_object_copies":18558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8783,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2787},{"shards":"2","objects":5757}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10629","reported_seq":17715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.423377+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:01.423377+0000","last_peered":"2024-09-11T05:18:01.423377+0000","last_clean":"2024-09-11T05:18:01.423377+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:01.423377+0000","last_undegraded":"2024-09-11T05:18:01.423377+0000","last_fullsized":"2024-09-11T05:18:01.423377+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37187584,"num_objects":9079,"num_object_clones":0,"num_object_copies":18158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8532,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10854","reported_seq":10986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.899376+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:00.899376+0000","last_peered":"2024-09-11T05:18:00.899376+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:00.899376+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7887,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":2842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10606","reported_seq":10740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.926976+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:00.926976+0000","last_peered":"2024-09-11T05:18:00.926976+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:00.926976+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37060608,"num_objects":9048,"num_object_clones":0,"num_object_copies":18096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7472,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5553},{"shards":"0,2","objects":2720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10677","reported_seq":10812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.928449+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:00.928449+0000","last_peered":"2024-09-11T05:18:00.928449+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:00.928449+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7435,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10598","reported_seq":10742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.856717+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:00.856717+0000","last_peered":"2024-09-11T05:18:00.856717+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:00.856717+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36831849,"num_objects":8993,"num_object_clones":0,"num_object_copies":17986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7605,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5478},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":601723521,"num_objects":146907,"num_object_clones":0,"num_object_copies":293814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":146907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134788,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5358,"num_bytes_recovered":21929984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1256767488,"data_stored":1256686285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656328,"kb_used_data":408056,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93715512,"statfs":{"total":96636764160,"available":95964684288,"internally_reserved":0,"allocated":417849344,"data_stored":417590418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593492,"kb_used_data":323012,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93778348,"statfs":{"total":96636764160,"available":96029028352,"internally_reserved":0,"allocated":330764288,"data_stored":330438663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215170,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849316,"kb_used_data":498384,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93522524,"statfs":{"total":96636764160,"available":95767064576,"internally_reserved":0,"allocated":510345216,"data_stored":510086285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359329451},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509161472,"data_stored":509157993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330182656,"data_stored":330112483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417423360,"data_stored":417415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:03.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:03.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10871", 'reported_seq': 10948, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.949603+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:00.949603+0000', 'last_peered': '2024-09-11T05:18:00.949603+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:00.949603+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38170624, 'num_objects': 9319, 'num_object_clones': 0, 'num_object_copies': 18638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10871, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5644}, {'shards': '0,2', 'objects': 2898}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10639", 'reported_seq': 10765, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:01.401955+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:01.401955+0000', 'last_peered': '2024-09-11T05:18:01.401955+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:01.401955+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37183512, 'num_objects': 9079, 'num_object_clones': 0, 'num_object_copies': 18158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9079, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9042, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2716}, {'shards': '2', 'objects': 5585}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10776", 'reported_seq': 14679, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.922993+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:00.922993+0000', 'last_peered': '2024-09-11T05:18:00.922993+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:00.922993+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37924864, 'num_objects': 9259, 'num_object_clones': 0, 'num_object_copies': 18518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1723, 'num_objects_unfound': 0, 'num_objects_dirty': 9259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7320, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1877, 'num_bytes_recovered': 7680000, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7534}, {'shards': '0,2', 'objects': 1723}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10912", 'reported_seq': 11045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.876270+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:00.876270+0000', 'last_peered': '2024-09-11T05:18:00.876270+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:00.876270+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38051840, 'num_objects': 9290, 'num_object_clones': 0, 'num_object_copies': 18580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8077, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5714}, {'shards': '0,2', 'objects': 2768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10767", 'reported_seq': 10899, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:01.396173+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:01.396173+0000', 'last_peered': '2024-09-11T05:18:01.396173+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:01.396173+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37699584, 'num_objects': 9204, 'num_object_clones': 0, 'num_object_copies': 18408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8835, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2747}, {'shards': '2', 'objects': 5679}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10572", 'reported_seq': 10737, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.754021+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:00.754021+0000', 'last_peered': '2024-09-11T05:18:00.754021+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:00.754021+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37273600, 'num_objects': 9100, 'num_object_clones': 0, 'num_object_copies': 18200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9100, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7325, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5658}, {'shards': '0,2', 'objects': 2710}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10754", 'reported_seq': 10872, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:01.422589+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:01.422589+0000', 'last_peered': '2024-09-11T05:18:01.422589+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:01.422589+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38006784, 'num_objects': 9279, 'num_object_clones': 0, 'num_object_copies': 18558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8783, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2787}, {'shards': '2', 'objects': 5757}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10854", 'reported_seq': 10986, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.899376+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:00.899376+0000', 'last_peered': '2024-09-11T05:18:00.899376+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:00.899376+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38051840, 'num_objects': 9290, 'num_object_clones': 0, 'num_object_copies': 18580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7887, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5667}, {'shards': '0,2', 'objects': 2842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10606", 'reported_seq': 10740, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.926976+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:00.926976+0000', 'last_peered': '2024-09-11T05:18:00.926976+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:00.926976+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37060608, 'num_objects': 9048, 'num_object_clones': 0, 'num_object_copies': 18096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7472, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5553}, {'shards': '0,2', 'objects': 2720}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10677", 'reported_seq': 10812, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.928449+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:00.928449+0000', 'last_peered': '2024-09-11T05:18:00.928449+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:00.928449+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37367808, 'num_objects': 9123, 'num_object_clones': 0, 'num_object_copies': 18246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7435, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5579}, {'shards': '0,2', 'objects': 2768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:03.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10598", 'reported_seq': 10742, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:00.856717+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:00.856717+0000', 'last_peered': '2024-09-11T05:18:00.856717+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:00.856717+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36831849, 'num_objects': 8993, 'num_object_clones': 0, 'num_object_copies': 17986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8993, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7605, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5478}, {'shards': '0,2', 'objects': 2718}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:03.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:04.615 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:04.616 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/293818 objects degraded (12.059%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":362,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":146909,"data_bytes":601723540,"bytes_used":2149515264,"bytes_avail":287760777216,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":293818,"degraded_ratio":0.12058825531451443,"misplaced_objects":1723,"misplaced_total":293818,"misplaced_ratio":0.0058641744209000125,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59355,"recovering_keys_per_sec":0,"num_objects_recovered":176,"num_bytes_recovered":712704,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:04.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:04.650 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:05.571 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:05.573 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:05.627 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-11T05:18:05.497454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601723540,"num_objects":146909,"num_object_clones":0,"num_object_copies":293818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":146909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134790,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5359,"num_bytes_recovered":21930003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2099136,"kb_used_data":1229452,"kb_used_omap":12,"kb_used_meta":869619,"kb_avail":281016384,"statfs":{"total":289910292480,"available":287760777216,"internally_reserved":0,"allocated":1258958848,"data_stored":1258115366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":890489857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,11,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18296832,"num_objects":-4467,"num_object_clones":0,"num_object_copies":-8934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-174,"num_objects_unfound":0,"num_objects_dirty":-4467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4467,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":712704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007250"},"pg_stats":[{"pgid":"2.f","version":"45'10871","reported_seq":10948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.949603+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:00.949603+0000","last_peered":"2024-09-11T05:18:00.949603+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:00.949603+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38170624,"num_objects":9319,"num_object_clones":0,"num_object_copies":18638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10871,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5644},{"shards":"0,2","objects":2898}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10737","reported_seq":10840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.378258+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:01.378258+0000","last_peered":"2024-09-11T05:18:01.378258+0000","last_clean":"2024-09-11T05:18:01.378258+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:01.378258+0000","last_undegraded":"2024-09-11T05:18:01.378258+0000","last_fullsized":"2024-09-11T05:18:01.378258+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37531648,"num_objects":9163,"num_object_clones":0,"num_object_copies":18326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9135,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10639","reported_seq":10765,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.401955+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:01.401955+0000","last_peered":"2024-09-11T05:18:01.401955+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:01.401955+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37183512,"num_objects":9079,"num_object_clones":0,"num_object_copies":18158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9042,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2716},{"shards":"2","objects":5585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10755","reported_seq":10827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.395401+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:01.395401+0000","last_peered":"2024-09-11T05:18:01.395401+0000","last_clean":"2024-09-11T05:18:01.395401+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:01.395401+0000","last_undegraded":"2024-09-11T05:18:01.395401+0000","last_fullsized":"2024-09-11T05:18:01.395401+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37703680,"num_objects":9205,"num_object_clones":0,"num_object_copies":18410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10755,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10831","reported_seq":10961,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:00.898329+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:00.898329+0000","last_peered":"2024-09-11T05:18:00.898329+0000","last_clean":"2024-09-11T05:18:00.898329+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:00.898329+0000","last_undegraded":"2024-09-11T05:18:00.898329+0000","last_fullsized":"2024-09-11T05:18:00.898329+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10776","reported_seq":14679,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.922993+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:00.922993+0000","last_peered":"2024-09-11T05:18:00.922993+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:00.922993+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7320,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":7680000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7534},{"shards":"0,2","objects":1723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10912","reported_seq":11045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.876270+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:00.876270+0000","last_peered":"2024-09-11T05:18:00.876270+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:00.876270+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8077,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10746","reported_seq":10870,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:00.926732+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:00.926732+0000","last_peered":"2024-09-11T05:18:00.926732+0000","last_clean":"2024-09-11T05:18:00.926732+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:00.926732+0000","last_undegraded":"2024-09-11T05:18:00.926732+0000","last_fullsized":"2024-09-11T05:18:00.926732+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8049,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10767","reported_seq":10899,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.396173+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:01.396173+0000","last_peered":"2024-09-11T05:18:01.396173+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:01.396173+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8835,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2747},{"shards":"2","objects":5679}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10572","reported_seq":10737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.754021+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:00.754021+0000","last_peered":"2024-09-11T05:18:00.754021+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:00.754021+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37273600,"num_objects":9100,"num_object_clones":0,"num_object_copies":18200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":2710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10754","reported_seq":10872,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:01.422589+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:01.422589+0000","last_peered":"2024-09-11T05:18:01.422589+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:01.422589+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38006784,"num_objects":9279,"num_object_clones":0,"num_object_copies":18558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8783,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2787},{"shards":"2","objects":5757}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10629","reported_seq":17715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:01.423377+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:01.423377+0000","last_peered":"2024-09-11T05:18:01.423377+0000","last_clean":"2024-09-11T05:18:01.423377+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:01.423377+0000","last_undegraded":"2024-09-11T05:18:01.423377+0000","last_fullsized":"2024-09-11T05:18:01.423377+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37187584,"num_objects":9079,"num_object_clones":0,"num_object_copies":18158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8532,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10854","reported_seq":10986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.899376+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:00.899376+0000","last_peered":"2024-09-11T05:18:00.899376+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:00.899376+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7887,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":2842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10606","reported_seq":10740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.926976+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:00.926976+0000","last_peered":"2024-09-11T05:18:00.926976+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:00.926976+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37060608,"num_objects":9048,"num_object_clones":0,"num_object_copies":18096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7472,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5553},{"shards":"0,2","objects":2720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10677","reported_seq":10812,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.928449+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:00.928449+0000","last_peered":"2024-09-11T05:18:00.928449+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:00.928449+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7435,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10598","reported_seq":10742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:00.856717+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:00.856717+0000","last_peered":"2024-09-11T05:18:00.856717+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:00.856717+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36831849,"num_objects":8993,"num_object_clones":0,"num_object_copies":17986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7605,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5478},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":601723521,"num_objects":146907,"num_object_clones":0,"num_object_copies":293814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1723,"num_objects_unfound":0,"num_objects_dirty":146907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134788,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5358,"num_bytes_recovered":21929984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1256767488,"data_stored":1256686285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656328,"kb_used_data":408056,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93715512,"statfs":{"total":96636764160,"available":95964684288,"internally_reserved":0,"allocated":417849344,"data_stored":417590418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593492,"kb_used_data":323012,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93778348,"statfs":{"total":96636764160,"available":96029028352,"internally_reserved":0,"allocated":330764288,"data_stored":330438663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215170,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849316,"kb_used_data":498384,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93522524,"statfs":{"total":96636764160,"available":95767064576,"internally_reserved":0,"allocated":510345216,"data_stored":510086285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359329451},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509161472,"data_stored":509157993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330182656,"data_stored":330112483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417423360,"data_stored":417415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:08.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:09.409 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-11T05:18:07.497935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592556692,"num_objects":144671,"num_object_clones":0,"num_object_copies":289342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1651,"num_objects_unfound":0,"num_objects_dirty":144671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137028,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5431,"num_bytes_recovered":22224915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079592,"kb_used_data":1218220,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":281035928,"statfs":{"total":289910292480,"available":287780790272,"internally_reserved":0,"allocated":1247457280,"data_stored":1246613798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":881970177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,4,9,10,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27463680,"num_objects":-6705,"num_object_clones":0,"num_object_copies":-13410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":-6705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6705,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007189"},"pg_stats":[{"pgid":"2.f","version":"45'11014","reported_seq":11093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.930891+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:05.930891+0000","last_peered":"2024-09-11T05:18:05.930891+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:05.930891+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37584896,"num_objects":9176,"num_object_clones":0,"num_object_copies":18352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5501},{"shards":"0,2","objects":2755}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10885","reported_seq":10988,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:06.366992+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:06.366992+0000","last_peered":"2024-09-11T05:18:06.366992+0000","last_clean":"2024-09-11T05:18:06.366992+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:06.366992+0000","last_undegraded":"2024-09-11T05:18:06.366992+0000","last_fullsized":"2024-09-11T05:18:06.366992+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9283,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10766","reported_seq":10892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:06.458618+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:06.458618+0000","last_peered":"2024-09-11T05:18:06.458618+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:06.458618+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663320,"num_objects":8952,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9169,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2589},{"shards":"2","objects":5458}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10913","reported_seq":10985,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:06.379561+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:06.379561+0000","last_peered":"2024-09-11T05:18:06.379561+0000","last_clean":"2024-09-11T05:18:06.379561+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:06.379561+0000","last_undegraded":"2024-09-11T05:18:06.379561+0000","last_fullsized":"2024-09-11T05:18:06.379561+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37056512,"num_objects":9047,"num_object_clones":0,"num_object_copies":18094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10913,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10984","reported_seq":11114,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:05.969861+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:05.969861+0000","last_peered":"2024-09-11T05:18:05.969861+0000","last_clean":"2024-09-11T05:18:05.969861+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:05.969861+0000","last_undegraded":"2024-09-11T05:18:05.969861+0000","last_fullsized":"2024-09-11T05:18:05.969861+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7818,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10947","reported_seq":14993,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.996861+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:05.996861+0000","last_peered":"2024-09-11T05:18:05.996861+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:05.996861+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37224448,"num_objects":9088,"num_object_clones":0,"num_object_copies":18176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1651,"num_objects_unfound":0,"num_objects_dirty":9088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7491,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":7974912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7435},{"shards":"0,2","objects":1651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11047","reported_seq":11180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.944022+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:05.944022+0000","last_peered":"2024-09-11T05:18:05.944022+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:05.944022+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37498880,"num_objects":9155,"num_object_clones":0,"num_object_copies":18310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8212,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":2633}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10880","reported_seq":11004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:05.936670+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:05.936670+0000","last_peered":"2024-09-11T05:18:05.936670+0000","last_clean":"2024-09-11T05:18:05.936670+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:05.936670+0000","last_undegraded":"2024-09-11T05:18:05.936670+0000","last_fullsized":"2024-09-11T05:18:05.936670+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8183,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10916","reported_seq":11048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:06.368437+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:06.368437+0000","last_peered":"2024-09-11T05:18:06.368437+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:06.368437+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37089280,"num_objects":9055,"num_object_clones":0,"num_object_copies":18110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8984,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2598},{"shards":"2","objects":5530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10711","reported_seq":10876,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.976636+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:05.976636+0000","last_peered":"2024-09-11T05:18:05.976636+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:05.976636+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704256,"num_objects":8961,"num_object_clones":0,"num_object_copies":17922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7464,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5519},{"shards":"0,2","objects":2571}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10873","reported_seq":10991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:06.420080+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:06.420080+0000","last_peered":"2024-09-11T05:18:06.420080+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:06.420080+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2668},{"shards":"2","objects":5638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10753","reported_seq":17839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:06.419812+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:06.419812+0000","last_peered":"2024-09-11T05:18:06.419812+0000","last_clean":"2024-09-11T05:18:06.419812+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:06.419812+0000","last_undegraded":"2024-09-11T05:18:06.419812+0000","last_fullsized":"2024-09-11T05:18:06.419812+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36679680,"num_objects":8955,"num_object_clones":0,"num_object_copies":17910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8656,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10995","reported_seq":11127,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.938163+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:05.938163+0000","last_peered":"2024-09-11T05:18:05.938163+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:05.938163+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8028,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5526},{"shards":"0,2","objects":2701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10743","reported_seq":10877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.946506+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:05.946506+0000","last_peered":"2024-09-11T05:18:05.946506+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:05.946506+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5416},{"shards":"0,2","objects":2583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10807","reported_seq":10942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.823892+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:05.823892+0000","last_peered":"2024-09-11T05:18:05.823892+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:05.823892+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36835328,"num_objects":8993,"num_object_clones":0,"num_object_copies":17986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7565,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5449},{"shards":"0,2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10728","reported_seq":10872,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.968898+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:05.968898+0000","last_peered":"2024-09-11T05:18:05.968898+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:05.968898+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36299369,"num_objects":8863,"num_object_clones":0,"num_object_copies":17726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7735,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5348},{"shards":"0,2","objects":2588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":592556673,"num_objects":144669,"num_object_clones":0,"num_object_copies":289338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1651,"num_objects_unfound":0,"num_objects_dirty":144669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137026,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5430,"num_bytes_recovered":22224896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245163520,"data_stored":1245082317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656084,"kb_used_data":403652,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93715756,"statfs":{"total":96636764160,"available":95964934144,"internally_reserved":0,"allocated":413339648,"data_stored":413080722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593492,"kb_used_data":323012,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93778348,"statfs":{"total":96636764160,"available":96029028352,"internally_reserved":0,"allocated":330764288,"data_stored":330438663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215171,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":830016,"kb_used_data":491556,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93541824,"statfs":{"total":96636764160,"available":95786827776,"internally_reserved":0,"allocated":503353344,"data_stored":503094413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502087680,"data_stored":502084201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330182656,"data_stored":330112483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412893184,"data_stored":412885633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:09.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:10.284 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:10.314 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-11T05:18:09.498447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592556692,"num_objects":144671,"num_object_clones":0,"num_object_copies":289342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1651,"num_objects_unfound":0,"num_objects_dirty":144671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137028,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5431,"num_bytes_recovered":22224915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2070352,"kb_used_data":1217300,"kb_used_omap":12,"kb_used_meta":852979,"kb_avail":281045168,"statfs":{"total":289910292480,"available":287790252032,"internally_reserved":0,"allocated":1246515200,"data_stored":1245671718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":873450497},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,6,8,10,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18247680,"num_objects":-4455,"num_object_clones":0,"num_object_copies":-8910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-152,"num_objects_unfound":0,"num_objects_dirty":-4455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":622592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007273"},"pg_stats":[{"pgid":"2.f","version":"45'11014","reported_seq":11093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.930891+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:05.930891+0000","last_peered":"2024-09-11T05:18:05.930891+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:05.930891+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37584896,"num_objects":9176,"num_object_clones":0,"num_object_copies":18352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5501},{"shards":"0,2","objects":2755}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10885","reported_seq":10988,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:06.366992+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:06.366992+0000","last_peered":"2024-09-11T05:18:06.366992+0000","last_clean":"2024-09-11T05:18:06.366992+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:06.366992+0000","last_undegraded":"2024-09-11T05:18:06.366992+0000","last_fullsized":"2024-09-11T05:18:06.366992+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9283,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10766","reported_seq":10892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:06.458618+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:06.458618+0000","last_peered":"2024-09-11T05:18:06.458618+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:06.458618+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663320,"num_objects":8952,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8952,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9169,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2589},{"shards":"2","objects":5458}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10913","reported_seq":10985,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:06.379561+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:06.379561+0000","last_peered":"2024-09-11T05:18:06.379561+0000","last_clean":"2024-09-11T05:18:06.379561+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:06.379561+0000","last_undegraded":"2024-09-11T05:18:06.379561+0000","last_fullsized":"2024-09-11T05:18:06.379561+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37056512,"num_objects":9047,"num_object_clones":0,"num_object_copies":18094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10913,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10984","reported_seq":11114,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:05.969861+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:05.969861+0000","last_peered":"2024-09-11T05:18:05.969861+0000","last_clean":"2024-09-11T05:18:05.969861+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:05.969861+0000","last_undegraded":"2024-09-11T05:18:05.969861+0000","last_fullsized":"2024-09-11T05:18:05.969861+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7818,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10947","reported_seq":14993,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.996861+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:05.996861+0000","last_peered":"2024-09-11T05:18:05.996861+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:05.996861+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37224448,"num_objects":9088,"num_object_clones":0,"num_object_copies":18176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1651,"num_objects_unfound":0,"num_objects_dirty":9088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7491,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":7974912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7435},{"shards":"0,2","objects":1651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11047","reported_seq":11180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.944022+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:05.944022+0000","last_peered":"2024-09-11T05:18:05.944022+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:05.944022+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37498880,"num_objects":9155,"num_object_clones":0,"num_object_copies":18310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8212,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":2633}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10880","reported_seq":11004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:05.936670+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:05.936670+0000","last_peered":"2024-09-11T05:18:05.936670+0000","last_clean":"2024-09-11T05:18:05.936670+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:05.936670+0000","last_undegraded":"2024-09-11T05:18:05.936670+0000","last_fullsized":"2024-09-11T05:18:05.936670+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8183,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10916","reported_seq":11048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:06.368437+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:06.368437+0000","last_peered":"2024-09-11T05:18:06.368437+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:06.368437+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37089280,"num_objects":9055,"num_object_clones":0,"num_object_copies":18110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8984,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2598},{"shards":"2","objects":5530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10711","reported_seq":10876,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.976636+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:05.976636+0000","last_peered":"2024-09-11T05:18:05.976636+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:05.976636+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704256,"num_objects":8961,"num_object_clones":0,"num_object_copies":17922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7464,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5519},{"shards":"0,2","objects":2571}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10873","reported_seq":10991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:06.420080+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:06.420080+0000","last_peered":"2024-09-11T05:18:06.420080+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:06.420080+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2668},{"shards":"2","objects":5638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10753","reported_seq":17839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:06.419812+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:06.419812+0000","last_peered":"2024-09-11T05:18:06.419812+0000","last_clean":"2024-09-11T05:18:06.419812+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:06.419812+0000","last_undegraded":"2024-09-11T05:18:06.419812+0000","last_fullsized":"2024-09-11T05:18:06.419812+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36679680,"num_objects":8955,"num_object_clones":0,"num_object_copies":17910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8656,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10995","reported_seq":11127,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.938163+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:05.938163+0000","last_peered":"2024-09-11T05:18:05.938163+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:05.938163+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8028,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5526},{"shards":"0,2","objects":2701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10743","reported_seq":10877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.946506+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:05.946506+0000","last_peered":"2024-09-11T05:18:05.946506+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:05.946506+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5416},{"shards":"0,2","objects":2583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10807","reported_seq":10942,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.823892+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:05.823892+0000","last_peered":"2024-09-11T05:18:05.823892+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:05.823892+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36835328,"num_objects":8993,"num_object_clones":0,"num_object_copies":17986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7565,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5449},{"shards":"0,2","objects":2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10728","reported_seq":10872,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:05.968898+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:05.968898+0000","last_peered":"2024-09-11T05:18:05.968898+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:05.968898+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36299369,"num_objects":8863,"num_object_clones":0,"num_object_copies":17726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7735,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5348},{"shards":"0,2","objects":2588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":592556673,"num_objects":144669,"num_object_clones":0,"num_object_copies":289338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1651,"num_objects_unfound":0,"num_objects_dirty":144669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137026,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5430,"num_bytes_recovered":22224896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1244295168,"data_stored":1244213965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656084,"kb_used_data":403652,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93715756,"statfs":{"total":96636764160,"available":95964934144,"internally_reserved":0,"allocated":413339648,"data_stored":413080722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258469547},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584252,"kb_used_data":322092,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93787588,"statfs":{"total":96636764160,"available":96038490112,"internally_reserved":0,"allocated":329822208,"data_stored":329496583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268431019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215171,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":830016,"kb_used_data":491556,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93541824,"statfs":{"total":96636764160,"available":95786827776,"internally_reserved":0,"allocated":503353344,"data_stored":503094413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502087680,"data_stored":502084201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329314304,"data_stored":329244131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412893184,"data_stored":412885633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:10.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:10.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11014", 'reported_seq': 11093, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.930891+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:05.930891+0000', 'last_peered': '2024-09-11T05:18:05.930891+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:05.930891+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37584896, 'num_objects': 9176, 'num_object_clones': 0, 'num_object_copies': 18352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9176, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11014, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5501}, {'shards': '0,2', 'objects': 2755}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:10.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10766", 'reported_seq': 10892, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:06.458618+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:06.458618+0000', 'last_peered': '2024-09-11T05:18:06.458618+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:06.458618+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36663320, 'num_objects': 8952, 'num_object_clones': 0, 'num_object_copies': 17904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8952, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9169, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2589}, {'shards': '2', 'objects': 5458}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10947", 'reported_seq': 14993, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.996861+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:05.996861+0000', 'last_peered': '2024-09-11T05:18:05.996861+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:05.996861+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37224448, 'num_objects': 9088, 'num_object_clones': 0, 'num_object_copies': 18176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1651, 'num_objects_unfound': 0, 'num_objects_dirty': 9088, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7491, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1949, 'num_bytes_recovered': 7974912, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7435}, {'shards': '0,2', 'objects': 1651}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11047", 'reported_seq': 11180, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.944022+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:05.944022+0000', 'last_peered': '2024-09-11T05:18:05.944022+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:05.944022+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37498880, 'num_objects': 9155, 'num_object_clones': 0, 'num_object_copies': 18310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8212, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5579}, {'shards': '0,2', 'objects': 2633}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10916", 'reported_seq': 11048, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:06.368437+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:06.368437+0000', 'last_peered': '2024-09-11T05:18:06.368437+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:06.368437+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37089280, 'num_objects': 9055, 'num_object_clones': 0, 'num_object_copies': 18110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8984, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2598}, {'shards': '2', 'objects': 5530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10711", 'reported_seq': 10876, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.976636+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:05.976636+0000', 'last_peered': '2024-09-11T05:18:05.976636+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:05.976636+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36704256, 'num_objects': 8961, 'num_object_clones': 0, 'num_object_copies': 17922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7464, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5519}, {'shards': '0,2', 'objects': 2571}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10873", 'reported_seq': 10991, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:06.420080+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:06.420080+0000', 'last_peered': '2024-09-11T05:18:06.420080+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:06.420080+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37519360, 'num_objects': 9160, 'num_object_clones': 0, 'num_object_copies': 18320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8902, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2668}, {'shards': '2', 'objects': 5638}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10995", 'reported_seq': 11127, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.938163+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:05.938163+0000', 'last_peered': '2024-09-11T05:18:05.938163+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:05.938163+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37474304, 'num_objects': 9149, 'num_object_clones': 0, 'num_object_copies': 18298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8028, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5526}, {'shards': '0,2', 'objects': 2701}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10743", 'reported_seq': 10877, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.946506+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:05.946506+0000', 'last_peered': '2024-09-11T05:18:05.946506+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:05.946506+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36499456, 'num_objects': 8911, 'num_object_clones': 0, 'num_object_copies': 17822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8911, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7609, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5416}, {'shards': '0,2', 'objects': 2583}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10807", 'reported_seq': 10942, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.823892+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:05.823892+0000', 'last_peered': '2024-09-11T05:18:05.823892+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:05.823892+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36835328, 'num_objects': 8993, 'num_object_clones': 0, 'num_object_copies': 17986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8993, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7565, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5449}, {'shards': '0,2', 'objects': 2638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:10.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10728", 'reported_seq': 10872, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:05.968898+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:05.968898+0000', 'last_peered': '2024-09-11T05:18:05.968898+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:05.968898+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36299369, 'num_objects': 8863, 'num_object_clones': 0, 'num_object_copies': 17726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7735, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5348}, {'shards': '0,2', 'objects': 2588}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:10.318 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:11.379 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/289342 objects degraded (12.245%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":369,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":144671,"data_bytes":592556692,"bytes_used":2120040448,"bytes_avail":287790252032,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":289342,"degraded_ratio":0.12245370530375817,"misplaced_objects":1651,"misplaced_total":289342,"misplaced_ratio":0.0057060502795999194,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":51851,"recovering_keys_per_sec":0,"num_objects_recovered":152,"num_bytes_recovered":622592,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":371},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:11.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:11.433 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:12.326 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:12.375 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-11T05:18:11.499746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583348884,"num_objects":142423,"num_object_clones":0,"num_object_copies":284846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":142423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139276,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5520,"num_bytes_recovered":22589459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2067472,"kb_used_data":1206112,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":281048048,"statfs":{"total":289910292480,"available":287793201152,"internally_reserved":0,"allocated":1235058688,"data_stored":1234215206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":881970177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,7,7,6,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27455488,"num_objects":-6703,"num_object_clones":0,"num_object_copies":-13406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":-6703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6703,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008176"},"pg_stats":[{"pgid":"2.f","version":"45'11179","reported_seq":11256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.028791+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:11.028791+0000","last_peered":"2024-09-11T05:18:11.028791+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:11.028791+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11179,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5336},{"shards":"0,2","objects":2590}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11008","reported_seq":11111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.436828+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:11.436828+0000","last_peered":"2024-09-11T05:18:11.436828+0000","last_clean":"2024-09-11T05:18:11.436828+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:11.436828+0000","last_undegraded":"2024-09-11T05:18:11.436828+0000","last_fullsized":"2024-09-11T05:18:11.436828+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36421632,"num_objects":8892,"num_object_clones":0,"num_object_copies":17784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10872","reported_seq":10998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.364769+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:11.364769+0000","last_peered":"2024-09-11T05:18:11.364769+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:11.364769+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229144,"num_objects":8846,"num_object_clones":0,"num_object_copies":17692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9275,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2483},{"shards":"2","objects":5352}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11062","reported_seq":11134,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.390044+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:11.390044+0000","last_peered":"2024-09-11T05:18:11.390044+0000","last_clean":"2024-09-11T05:18:11.390044+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:11.390044+0000","last_undegraded":"2024-09-11T05:18:11.390044+0000","last_fullsized":"2024-09-11T05:18:11.390044+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36446208,"num_objects":8898,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11062,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11146","reported_seq":11275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.028836+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:11.028836+0000","last_peered":"2024-09-11T05:18:11.028836+0000","last_clean":"2024-09-11T05:18:11.028836+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:11.028836+0000","last_undegraded":"2024-09-11T05:18:11.028836+0000","last_fullsized":"2024-09-11T05:18:11.028836+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36958208,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7980,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11100","reported_seq":15325,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.032473+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:11.032473+0000","last_peered":"2024-09-11T05:18:11.032473+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:11.032473+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7644,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2038,"num_bytes_recovered":8339456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7371},{"shards":"0,2","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11185","reported_seq":11318,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:10.995168+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:10.995168+0000","last_peered":"2024-09-11T05:18:10.995168+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:10.995168+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8350,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5441},{"shards":"0,2","objects":2495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11042","reported_seq":11166,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.043398+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:11.043398+0000","last_peered":"2024-09-11T05:18:11.043398+0000","last_clean":"2024-09-11T05:18:11.043398+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:11.043398+0000","last_undegraded":"2024-09-11T05:18:11.043398+0000","last_fullsized":"2024-09-11T05:18:11.043398+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8345,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.569979+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:15:57.569979+0000","last_peered":"2024-09-11T05:15:57.569979+0000","last_clean":"2024-09-11T05:15:57.569979+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:15:57.569979+0000","last_undegraded":"2024-09-11T05:15:57.569979+0000","last_fullsized":"2024-09-11T05:15:57.569979+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11048","reported_seq":11179,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.400885+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:11.400885+0000","last_peered":"2024-09-11T05:18:11.400885+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:11.400885+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9116,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2466},{"shards":"2","objects":5398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10843","reported_seq":11008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.016464+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:11.016464+0000","last_peered":"2024-09-11T05:18:11.016464+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:11.016464+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36163584,"num_objects":8829,"num_object_clones":0,"num_object_copies":17658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5387},{"shards":"0,2","objects":2439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11012","reported_seq":11130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.436416+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:11.436416+0000","last_peered":"2024-09-11T05:18:11.436416+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:11.436416+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36950016,"num_objects":9021,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2529},{"shards":"2","objects":5499}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10886","reported_seq":17972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.326248+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:11.326248+0000","last_peered":"2024-09-11T05:18:11.326248+0000","last_clean":"2024-09-11T05:18:11.326248+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:11.326248+0000","last_undegraded":"2024-09-11T05:18:11.326248+0000","last_fullsized":"2024-09-11T05:18:11.326248+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8789,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11153","reported_seq":11284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.010621+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:11.010621+0000","last_peered":"2024-09-11T05:18:11.010621+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:11.010621+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8186,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5368},{"shards":"0,2","objects":2543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10865","reported_seq":10997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.010718+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:11.010718+0000","last_peered":"2024-09-11T05:18:11.010718+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:11.010718+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35999744,"num_objects":8789,"num_object_clones":0,"num_object_copies":17578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7731,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5294},{"shards":"0,2","objects":2461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10949","reported_seq":11084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:10.969441+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:10.969441+0000","last_peered":"2024-09-11T05:18:10.969441+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:10.969441+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7707,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5307},{"shards":"0,2","objects":2496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10860","reported_seq":11004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.002278+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:11.002278+0000","last_peered":"2024-09-11T05:18:11.002278+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:11.002278+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758697,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7867,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":2456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583348865,"num_objects":142421,"num_object_clones":0,"num_object_copies":284842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":142421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139274,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5519,"num_bytes_recovered":22589440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1232707584,"data_stored":1232626381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655888,"kb_used_data":399296,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93715952,"statfs":{"total":96636764160,"available":95965134848,"internally_reserved":0,"allocated":408879104,"data_stored":408620178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584252,"kb_used_data":322092,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93787588,"statfs":{"total":96636764160,"available":96038490112,"internally_reserved":0,"allocated":329822208,"data_stored":329496583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268431019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215172,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":827332,"kb_used_data":484724,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93544508,"statfs":{"total":96636764160,"available":95789576192,"internally_reserved":0,"allocated":496357376,"data_stored":496098445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,3,5,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":494977024,"data_stored":494973545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329314304,"data_stored":329244131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408416256,"data_stored":408408705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:15.378 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:16.312 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:16.405 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-11T05:18:15.501085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583348884,"num_objects":142423,"num_object_clones":0,"num_object_copies":284846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":142423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139276,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5520,"num_bytes_recovered":22589459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2070760,"kb_used_data":1205240,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":281044760,"statfs":{"total":289910292480,"available":287789834240,"internally_reserved":0,"allocated":1234165760,"data_stored":1233322278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":886230017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,4,11,8,9,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18374656,"num_objects":-4486,"num_object_clones":0,"num_object_copies":-8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":-4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4486,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003994"},"pg_stats":[{"pgid":"2.f","version":"45'11179","reported_seq":11256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.028791+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:11.028791+0000","last_peered":"2024-09-11T05:18:11.028791+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:11.028791+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11179,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5336},{"shards":"0,2","objects":2590}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11008","reported_seq":11111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.436828+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:11.436828+0000","last_peered":"2024-09-11T05:18:11.436828+0000","last_clean":"2024-09-11T05:18:11.436828+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:11.436828+0000","last_undegraded":"2024-09-11T05:18:11.436828+0000","last_fullsized":"2024-09-11T05:18:11.436828+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36421632,"num_objects":8892,"num_object_clones":0,"num_object_copies":17784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10872","reported_seq":10998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.364769+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:11.364769+0000","last_peered":"2024-09-11T05:18:11.364769+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:11.364769+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229144,"num_objects":8846,"num_object_clones":0,"num_object_copies":17692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9275,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2483},{"shards":"2","objects":5352}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11062","reported_seq":11134,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.390044+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:11.390044+0000","last_peered":"2024-09-11T05:18:11.390044+0000","last_clean":"2024-09-11T05:18:11.390044+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:11.390044+0000","last_undegraded":"2024-09-11T05:18:11.390044+0000","last_fullsized":"2024-09-11T05:18:11.390044+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36446208,"num_objects":8898,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11062,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11146","reported_seq":11275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.028836+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:11.028836+0000","last_peered":"2024-09-11T05:18:11.028836+0000","last_clean":"2024-09-11T05:18:11.028836+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:11.028836+0000","last_undegraded":"2024-09-11T05:18:11.028836+0000","last_fullsized":"2024-09-11T05:18:11.028836+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36958208,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7980,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11100","reported_seq":15325,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.032473+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:11.032473+0000","last_peered":"2024-09-11T05:18:11.032473+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:11.032473+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7644,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2038,"num_bytes_recovered":8339456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7371},{"shards":"0,2","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11185","reported_seq":11318,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:10.995168+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:10.995168+0000","last_peered":"2024-09-11T05:18:10.995168+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:10.995168+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8350,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5441},{"shards":"0,2","objects":2495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11042","reported_seq":11166,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.043398+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:11.043398+0000","last_peered":"2024-09-11T05:18:11.043398+0000","last_clean":"2024-09-11T05:18:11.043398+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:11.043398+0000","last_undegraded":"2024-09-11T05:18:11.043398+0000","last_fullsized":"2024-09-11T05:18:11.043398+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8345,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11048","reported_seq":11179,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.400885+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:11.400885+0000","last_peered":"2024-09-11T05:18:11.400885+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:11.400885+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9116,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2466},{"shards":"2","objects":5398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10843","reported_seq":11008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.016464+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:11.016464+0000","last_peered":"2024-09-11T05:18:11.016464+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:11.016464+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36163584,"num_objects":8829,"num_object_clones":0,"num_object_copies":17658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5387},{"shards":"0,2","objects":2439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11012","reported_seq":11130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.436416+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:11.436416+0000","last_peered":"2024-09-11T05:18:11.436416+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:11.436416+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36950016,"num_objects":9021,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2529},{"shards":"2","objects":5499}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10886","reported_seq":17972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.326248+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:11.326248+0000","last_peered":"2024-09-11T05:18:11.326248+0000","last_clean":"2024-09-11T05:18:11.326248+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:11.326248+0000","last_undegraded":"2024-09-11T05:18:11.326248+0000","last_fullsized":"2024-09-11T05:18:11.326248+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8789,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11153","reported_seq":11284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.010621+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:11.010621+0000","last_peered":"2024-09-11T05:18:11.010621+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:11.010621+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8186,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5368},{"shards":"0,2","objects":2543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10865","reported_seq":10997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.010718+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:11.010718+0000","last_peered":"2024-09-11T05:18:11.010718+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:11.010718+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35999744,"num_objects":8789,"num_object_clones":0,"num_object_copies":17578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7731,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5294},{"shards":"0,2","objects":2461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10949","reported_seq":11084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:10.969441+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:10.969441+0000","last_peered":"2024-09-11T05:18:10.969441+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:10.969441+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7707,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5307},{"shards":"0,2","objects":2496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10860","reported_seq":11004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.002278+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:11.002278+0000","last_peered":"2024-09-11T05:18:11.002278+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:11.002278+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758697,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7867,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":2456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583348865,"num_objects":142421,"num_object_clones":0,"num_object_copies":284842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":142421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139274,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5519,"num_bytes_recovered":22589440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1231904768,"data_stored":1231823565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655888,"kb_used_data":399296,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93715952,"statfs":{"total":96636764160,"available":95965134848,"internally_reserved":0,"allocated":408879104,"data_stored":408620178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587540,"kb_used_data":321220,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93784300,"statfs":{"total":96636764160,"available":96035123200,"internally_reserved":0,"allocated":328929280,"data_stored":328603655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,6,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215172,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":827332,"kb_used_data":484724,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93544508,"statfs":{"total":96636764160,"available":95789576192,"internally_reserved":0,"allocated":496357376,"data_stored":496098445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,3,5,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":494977024,"data_stored":494973545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328511488,"data_stored":328441315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408416256,"data_stored":408408705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:16.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:17.263 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:17.264 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:17.302 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-11T05:18:15.501085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583348884,"num_objects":142423,"num_object_clones":0,"num_object_copies":284846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":142423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139276,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5520,"num_bytes_recovered":22589459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2070760,"kb_used_data":1205240,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":281044760,"statfs":{"total":289910292480,"available":287789834240,"internally_reserved":0,"allocated":1234165760,"data_stored":1233322278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":886230017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,4,11,8,9,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18374656,"num_objects":-4486,"num_object_clones":0,"num_object_copies":-8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":-4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4486,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003994"},"pg_stats":[{"pgid":"2.f","version":"45'11179","reported_seq":11256,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.028791+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:11.028791+0000","last_peered":"2024-09-11T05:18:11.028791+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:11.028791+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36909056,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11179,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5336},{"shards":"0,2","objects":2590}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11008","reported_seq":11111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.436828+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:11.436828+0000","last_peered":"2024-09-11T05:18:11.436828+0000","last_clean":"2024-09-11T05:18:11.436828+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:11.436828+0000","last_undegraded":"2024-09-11T05:18:11.436828+0000","last_fullsized":"2024-09-11T05:18:11.436828+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36421632,"num_objects":8892,"num_object_clones":0,"num_object_copies":17784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10872","reported_seq":10998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.364769+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:11.364769+0000","last_peered":"2024-09-11T05:18:11.364769+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:11.364769+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229144,"num_objects":8846,"num_object_clones":0,"num_object_copies":17692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9275,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2483},{"shards":"2","objects":5352}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11062","reported_seq":11134,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.390044+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:11.390044+0000","last_peered":"2024-09-11T05:18:11.390044+0000","last_clean":"2024-09-11T05:18:11.390044+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:11.390044+0000","last_undegraded":"2024-09-11T05:18:11.390044+0000","last_fullsized":"2024-09-11T05:18:11.390044+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36446208,"num_objects":8898,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11062,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11146","reported_seq":11275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.028836+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:11.028836+0000","last_peered":"2024-09-11T05:18:11.028836+0000","last_clean":"2024-09-11T05:18:11.028836+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:11.028836+0000","last_undegraded":"2024-09-11T05:18:11.028836+0000","last_fullsized":"2024-09-11T05:18:11.028836+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36958208,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7980,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11100","reported_seq":15325,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.032473+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:11.032473+0000","last_peered":"2024-09-11T05:18:11.032473+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:11.032473+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7644,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2038,"num_bytes_recovered":8339456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7371},{"shards":"0,2","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11185","reported_seq":11318,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:10.995168+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:10.995168+0000","last_peered":"2024-09-11T05:18:10.995168+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:10.995168+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8350,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5441},{"shards":"0,2","objects":2495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11042","reported_seq":11166,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.043398+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:11.043398+0000","last_peered":"2024-09-11T05:18:11.043398+0000","last_clean":"2024-09-11T05:18:11.043398+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:11.043398+0000","last_undegraded":"2024-09-11T05:18:11.043398+0000","last_fullsized":"2024-09-11T05:18:11.043398+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8345,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11048","reported_seq":11179,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.400885+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:11.400885+0000","last_peered":"2024-09-11T05:18:11.400885+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:11.400885+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9116,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2466},{"shards":"2","objects":5398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10843","reported_seq":11008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.016464+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:11.016464+0000","last_peered":"2024-09-11T05:18:11.016464+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:11.016464+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36163584,"num_objects":8829,"num_object_clones":0,"num_object_copies":17658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5387},{"shards":"0,2","objects":2439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11012","reported_seq":11130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.436416+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:11.436416+0000","last_peered":"2024-09-11T05:18:11.436416+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:11.436416+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36950016,"num_objects":9021,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2529},{"shards":"2","objects":5499}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10886","reported_seq":17972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:11.326248+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:11.326248+0000","last_peered":"2024-09-11T05:18:11.326248+0000","last_clean":"2024-09-11T05:18:11.326248+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:11.326248+0000","last_undegraded":"2024-09-11T05:18:11.326248+0000","last_fullsized":"2024-09-11T05:18:11.326248+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8789,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11153","reported_seq":11284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.010621+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:11.010621+0000","last_peered":"2024-09-11T05:18:11.010621+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:11.010621+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8186,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5368},{"shards":"0,2","objects":2543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10865","reported_seq":10997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.010718+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:11.010718+0000","last_peered":"2024-09-11T05:18:11.010718+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:11.010718+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35999744,"num_objects":8789,"num_object_clones":0,"num_object_copies":17578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7731,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5294},{"shards":"0,2","objects":2461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10949","reported_seq":11084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:10.969441+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:10.969441+0000","last_peered":"2024-09-11T05:18:10.969441+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:10.969441+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36253696,"num_objects":8851,"num_object_clones":0,"num_object_copies":17702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7707,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5307},{"shards":"0,2","objects":2496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10860","reported_seq":11004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:11.002278+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:11.002278+0000","last_peered":"2024-09-11T05:18:11.002278+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:11.002278+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758697,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7867,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":2456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583348865,"num_objects":142421,"num_object_clones":0,"num_object_copies":284842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1562,"num_objects_unfound":0,"num_objects_dirty":142421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139274,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5519,"num_bytes_recovered":22589440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1231904768,"data_stored":1231823565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655888,"kb_used_data":399296,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93715952,"statfs":{"total":96636764160,"available":95965134848,"internally_reserved":0,"allocated":408879104,"data_stored":408620178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":262729387},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587540,"kb_used_data":321220,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93784300,"statfs":{"total":96636764160,"available":96035123200,"internally_reserved":0,"allocated":328929280,"data_stored":328603655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,6,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215172,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":827332,"kb_used_data":484724,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93544508,"statfs":{"total":96636764160,"available":95789576192,"internally_reserved":0,"allocated":496357376,"data_stored":496098445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,3,5,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":494977024,"data_stored":494973545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328511488,"data_stored":328441315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408416256,"data_stored":408408705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:17.303 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:17.303 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11179", 'reported_seq': 11256, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.028791+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:11.028791+0000', 'last_peered': '2024-09-11T05:18:11.028791+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:11.028791+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36909056, 'num_objects': 9011, 'num_object_clones': 0, 'num_object_copies': 18022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9011, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11179, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5336}, {'shards': '0,2', 'objects': 2590}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10872", 'reported_seq': 10998, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.364769+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:11.364769+0000', 'last_peered': '2024-09-11T05:18:11.364769+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:11.364769+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36229144, 'num_objects': 8846, 'num_object_clones': 0, 'num_object_copies': 17692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8846, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9275, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2483}, {'shards': '2', 'objects': 5352}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11100", 'reported_seq': 15325, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.032473+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:11.032473+0000', 'last_peered': '2024-09-11T05:18:11.032473+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:11.032473+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36597760, 'num_objects': 8935, 'num_object_clones': 0, 'num_object_copies': 17870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1562, 'num_objects_unfound': 0, 'num_objects_dirty': 8935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7644, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2038, 'num_bytes_recovered': 8339456, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7371}, {'shards': '0,2', 'objects': 1562}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11185", 'reported_seq': 11318, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:10.995168+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:10.995168+0000', 'last_peered': '2024-09-11T05:18:10.995168+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:10.995168+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36933632, 'num_objects': 9017, 'num_object_clones': 0, 'num_object_copies': 18034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8350, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5441}, {'shards': '0,2', 'objects': 2495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11048", 'reported_seq': 11179, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.400885+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:11.400885+0000', 'last_peered': '2024-09-11T05:18:11.400885+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:11.400885+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36548608, 'num_objects': 8923, 'num_object_clones': 0, 'num_object_copies': 17846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9116, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2466}, {'shards': '2', 'objects': 5398}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:17.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10843", 'reported_seq': 11008, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.016464+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:11.016464+0000', 'last_peered': '2024-09-11T05:18:11.016464+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:11.016464+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36163584, 'num_objects': 8829, 'num_object_clones': 0, 'num_object_copies': 17658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7596, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5387}, {'shards': '0,2', 'objects': 2439}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11012", 'reported_seq': 11130, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.436416+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:11.436416+0000', 'last_peered': '2024-09-11T05:18:11.436416+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:11.436416+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36950016, 'num_objects': 9021, 'num_object_clones': 0, 'num_object_copies': 18042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9041, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2529}, {'shards': '2', 'objects': 5499}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11153", 'reported_seq': 11284, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.010621+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:11.010621+0000', 'last_peered': '2024-09-11T05:18:11.010621+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:11.010621+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36827136, 'num_objects': 8991, 'num_object_clones': 0, 'num_object_copies': 17982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8186, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5368}, {'shards': '0,2', 'objects': 2543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10865", 'reported_seq': 10997, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.010718+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:11.010718+0000', 'last_peered': '2024-09-11T05:18:11.010718+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:11.010718+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35999744, 'num_objects': 8789, 'num_object_clones': 0, 'num_object_copies': 17578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8789, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7731, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5294}, {'shards': '0,2', 'objects': 2461}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:17.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10949", 'reported_seq': 11084, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:10.969441+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:10.969441+0000', 'last_peered': '2024-09-11T05:18:10.969441+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:10.969441+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36253696, 'num_objects': 8851, 'num_object_clones': 0, 'num_object_copies': 17702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7707, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5307}, {'shards': '0,2', 'objects': 2496}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:17.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10860", 'reported_seq': 11004, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:11.002278+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:11.002278+0000', 'last_peered': '2024-09-11T05:18:11.002278+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:11.002278+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35758697, 'num_objects': 8731, 'num_object_clones': 0, 'num_object_copies': 17462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7867, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5216}, {'shards': '0,2', 'objects': 2456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:17.306 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:18.238 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:18.238 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/280314 objects degraded (12.640%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":376,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140157,"data_bytes":574067348,"bytes_used":2104492032,"bytes_avail":287805800448,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":280314,"degraded_ratio":0.12639753990168168,"misplaced_objects":1476,"misplaced_total":280314,"misplaced_ratio":0.0052655236627496306,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":84274,"recovering_keys_per_sec":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":562},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:18.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:18.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:19.223 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:19.229 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:19.274 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-11T05:18:17.502443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574067348,"num_objects":140157,"num_object_clones":0,"num_object_copies":280314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1476,"num_objects_unfound":0,"num_objects_dirty":140157,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141542,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5606,"num_bytes_recovered":22941715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2055168,"kb_used_data":1193796,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":281060352,"statfs":{"total":289910292480,"available":287805800448,"internally_reserved":0,"allocated":1222447104,"data_stored":1221603622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":881970177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,12,3,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27656192,"num_objects":-6752,"num_object_clones":0,"num_object_copies":-13504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-247,"num_objects_unfound":0,"num_objects_dirty":-6752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004989"},"pg_stats":[{"pgid":"2.f","version":"45'11320","reported_seq":11399,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:15.773176+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:15.773176+0000","last_peered":"2024-09-11T05:18:15.773176+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:15.773176+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36331520,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11320,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5195},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11163","reported_seq":11266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:16.477551+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:16.477551+0000","last_peered":"2024-09-11T05:18:16.477551+0000","last_clean":"2024-09-11T05:18:16.477551+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:16.477551+0000","last_undegraded":"2024-09-11T05:18:16.477551+0000","last_fullsized":"2024-09-11T05:18:16.477551+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9561,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11008","reported_seq":11134,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.478004+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:16.478004+0000","last_peered":"2024-09-11T05:18:16.478004+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:16.478004+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672088,"num_objects":8710,"num_object_clones":0,"num_object_copies":17420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9411,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2347},{"shards":"2","objects":5216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11216","reported_seq":11288,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:16.455282+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:16.455282+0000","last_peered":"2024-09-11T05:18:16.455282+0000","last_clean":"2024-09-11T05:18:16.455282+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:16.455282+0000","last_undegraded":"2024-09-11T05:18:16.455282+0000","last_fullsized":"2024-09-11T05:18:16.455282+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11216,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11290","reported_seq":11419,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:16.087320+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:16.087320+0000","last_peered":"2024-09-11T05:18:16.087320+0000","last_clean":"2024-09-11T05:18:16.087320+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:16.087320+0000","last_undegraded":"2024-09-11T05:18:16.087320+0000","last_fullsized":"2024-09-11T05:18:16.087320+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36368384,"num_objects":8879,"num_object_clones":0,"num_object_copies":17758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8124,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11247","reported_seq":15645,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.048672+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:16.048672+0000","last_peered":"2024-09-11T05:18:16.048672+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:16.048672+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1476,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7791,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2124,"num_bytes_recovered":8691712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7310},{"shards":"0,2","objects":1476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11322","reported_seq":11455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.027879+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:16.027879+0000","last_peered":"2024-09-11T05:18:16.027879+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:16.027879+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36372480,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8487,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5304},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11172","reported_seq":11296,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:15.996500+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:15.996500+0000","last_peered":"2024-09-11T05:18:15.996500+0000","last_clean":"2024-09-11T05:18:15.996500+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:15.996500+0000","last_undegraded":"2024-09-11T05:18:15.996500+0000","last_fullsized":"2024-09-11T05:18:15.996500+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8475,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11193","reported_seq":11325,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.409306+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:16.409306+0000","last_peered":"2024-09-11T05:18:16.409306+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:16.409306+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35954688,"num_objects":8778,"num_object_clones":0,"num_object_copies":17556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9261,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2321},{"shards":"2","objects":5253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10985","reported_seq":11149,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.068522+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:16.068522+0000","last_peered":"2024-09-11T05:18:16.068522+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:16.068522+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35581952,"num_objects":8687,"num_object_clones":0,"num_object_copies":17374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":2297}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11146","reported_seq":11264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.392527+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:16.392527+0000","last_peered":"2024-09-11T05:18:16.392527+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:16.392527+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9175,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2395},{"shards":"2","objects":5365}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11013","reported_seq":18099,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:16.429923+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:16.429923+0000","last_peered":"2024-09-11T05:18:16.429923+0000","last_clean":"2024-09-11T05:18:16.429923+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:16.429923+0000","last_undegraded":"2024-09-11T05:18:16.429923+0000","last_fullsized":"2024-09-11T05:18:16.429923+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35614720,"num_objects":8695,"num_object_clones":0,"num_object_copies":17390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8916,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11301","reported_seq":11432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.044707+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:16.044707+0000","last_peered":"2024-09-11T05:18:16.044707+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:16.044707+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8334,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5220},{"shards":"0,2","objects":2395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11002","reported_seq":11135,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.084499+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:16.084499+0000","last_peered":"2024-09-11T05:18:16.084499+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:16.084499+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35438592,"num_objects":8652,"num_object_clones":0,"num_object_copies":17304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7868,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5157},{"shards":"0,2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11083","reported_seq":11218,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:15.996981+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:15.996981+0000","last_peered":"2024-09-11T05:18:15.996981+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:15.996981+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35704832,"num_objects":8717,"num_object_clones":0,"num_object_copies":17434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7841,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5173},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11015","reported_seq":11159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:15.987938+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:15.987938+0000","last_peered":"2024-09-11T05:18:15.987938+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:15.987938+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35123817,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8022,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":2301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574067329,"num_objects":140155,"num_object_clones":0,"num_object_copies":280310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1476,"num_objects_unfound":0,"num_objects_dirty":140155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141540,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5605,"num_bytes_recovered":22941696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1220227072,"data_stored":1220145869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655508,"kb_used_data":394744,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93716332,"statfs":{"total":96636764160,"available":95965523968,"internally_reserved":0,"allocated":404217856,"data_stored":403958930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":266989227},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587540,"kb_used_data":321220,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93784300,"statfs":{"total":96636764160,"available":96035123200,"internally_reserved":0,"allocated":328929280,"data_stored":328603655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272690859},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,6,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215173,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":812120,"kb_used_data":477832,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93559720,"statfs":{"total":96636764160,"available":95805153280,"internally_reserved":0,"allocated":489299968,"data_stored":489041037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487927808,"data_stored":487924329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328511488,"data_stored":328441315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":403787776,"data_stored":403780225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:22.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:23.140 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:23.141 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:23.179 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-11T05:18:21.512693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568435348,"num_objects":138782,"num_object_clones":0,"num_object_copies":277564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":138782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142917,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5700,"num_bytes_recovered":23326739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2060392,"kb_used_data":1186540,"kb_used_omap":12,"kb_used_meta":873779,"kb_avail":281055128,"statfs":{"total":289910292480,"available":287800451072,"internally_reserved":0,"allocated":1215016960,"data_stored":1214173478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":894749697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,0,7,7,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24121344,"num_objects":-5889,"num_object_clones":0,"num_object_copies":-11778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-269,"num_objects_unfound":0,"num_objects_dirty":-5889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5889,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":1101824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.014245"},"pg_stats":[{"pgid":"2.f","version":"45'11467","reported_seq":11544,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.166961+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:21.166961+0000","last_peered":"2024-09-11T05:18:21.166961+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:21.166961+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5048},{"shards":"0,2","objects":2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11163","reported_seq":11266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:16.477551+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:16.477551+0000","last_peered":"2024-09-11T05:18:16.477551+0000","last_clean":"2024-09-11T05:18:16.477551+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:16.477551+0000","last_undegraded":"2024-09-11T05:18:16.477551+0000","last_fullsized":"2024-09-11T05:18:16.477551+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9561,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11008","reported_seq":11134,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.478004+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:16.478004+0000","last_peered":"2024-09-11T05:18:16.478004+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:16.478004+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672088,"num_objects":8710,"num_object_clones":0,"num_object_copies":17420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9411,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2347},{"shards":"2","objects":5216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11216","reported_seq":11288,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:16.455282+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:16.455282+0000","last_peered":"2024-09-11T05:18:16.455282+0000","last_clean":"2024-09-11T05:18:16.455282+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:16.455282+0000","last_undegraded":"2024-09-11T05:18:16.455282+0000","last_fullsized":"2024-09-11T05:18:16.455282+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11216,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11425","reported_seq":11555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.115454+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:21.115454+0000","last_peered":"2024-09-11T05:18:21.115454+0000","last_clean":"2024-09-11T05:18:21.115454+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:21.115454+0000","last_undegraded":"2024-09-11T05:18:21.115454+0000","last_fullsized":"2024-09-11T05:18:21.115454+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8259,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11380","reported_seq":15966,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.136842+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:21.136842+0000","last_peered":"2024-09-11T05:18:21.136842+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:21.136842+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2218,"num_bytes_recovered":9076736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7271},{"shards":"0,2","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11463","reported_seq":11596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.115811+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:21.115811+0000","last_peered":"2024-09-11T05:18:21.115811+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:21.115811+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35794944,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5163},{"shards":"0,2","objects":2217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11298","reported_seq":11422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.045747+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:21.045747+0000","last_peered":"2024-09-11T05:18:21.045747+0000","last_clean":"2024-09-11T05:18:21.045747+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:21.045747+0000","last_undegraded":"2024-09-11T05:18:21.045747+0000","last_fullsized":"2024-09-11T05:18:21.045747+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8601,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11193","reported_seq":11325,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.409306+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:16.409306+0000","last_peered":"2024-09-11T05:18:16.409306+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:16.409306+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35954688,"num_objects":8778,"num_object_clones":0,"num_object_copies":17556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9261,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2321},{"shards":"2","objects":5253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11132","reported_seq":11298,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.111917+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:21.111917+0000","last_peered":"2024-09-11T05:18:21.111917+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:21.111917+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34979840,"num_objects":8540,"num_object_clones":0,"num_object_copies":17080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7885,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5098},{"shards":"0,2","objects":2150}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11146","reported_seq":11264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:16.392527+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:16.392527+0000","last_peered":"2024-09-11T05:18:16.392527+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:16.392527+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9175,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2395},{"shards":"2","objects":5365}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570208+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:15:57.570208+0000","last_peered":"2024-09-11T05:15:57.570208+0000","last_clean":"2024-09-11T05:15:57.570208+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:15:57.570208+0000","last_undegraded":"2024-09-11T05:15:57.570208+0000","last_fullsized":"2024-09-11T05:15:57.570208+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11013","reported_seq":18099,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:16.429923+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:16.429923+0000","last_peered":"2024-09-11T05:18:16.429923+0000","last_clean":"2024-09-11T05:18:16.429923+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:16.429923+0000","last_undegraded":"2024-09-11T05:18:16.429923+0000","last_fullsized":"2024-09-11T05:18:16.429923+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35614720,"num_objects":8695,"num_object_clones":0,"num_object_copies":17390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8916,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11463","reported_seq":11595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.135722+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:21.135722+0000","last_peered":"2024-09-11T05:18:21.135722+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:21.135722+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35557376,"num_objects":8681,"num_object_clones":0,"num_object_copies":17362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8496,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570845+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:15:57.570845+0000","last_peered":"2024-09-11T05:15:57.570845+0000","last_clean":"2024-09-11T05:15:57.570845+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:15:57.570845+0000","last_undegraded":"2024-09-11T05:15:57.570845+0000","last_fullsized":"2024-09-11T05:15:57.570845+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11117","reported_seq":11250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.074346+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:21.074346+0000","last_peered":"2024-09-11T05:18:21.074346+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:21.074346+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34967552,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7983,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11224","reported_seq":11355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.165066+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:21.165066+0000","last_peered":"2024-09-11T05:18:21.165066+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:21.165066+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7982,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570645+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:15:57.570645+0000","last_peered":"2024-09-11T05:15:57.570645+0000","last_clean":"2024-09-11T05:15:57.570645+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:15:57.570645+0000","last_undegraded":"2024-09-11T05:15:57.570645+0000","last_fullsized":"2024-09-11T05:15:57.570645+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11143","reported_seq":11287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.138219+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:21.138219+0000","last_peered":"2024-09-11T05:18:21.138219+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:21.138219+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34599529,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4933},{"shards":"0,2","objects":2173}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568435329,"num_objects":138780,"num_object_clones":0,"num_object_copies":277560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":138780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142915,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5699,"num_bytes_recovered":23326720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212637184,"data_stored":1212555981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655508,"kb_used_data":394744,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93716332,"statfs":{"total":96636764160,"available":95965523968,"internally_reserved":0,"allocated":404217856,"data_stored":403958930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":266989227},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595036,"kb_used_data":320396,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93776804,"statfs":{"total":96636764160,"available":96027447296,"internally_reserved":0,"allocated":328085504,"data_stored":327759879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215174,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809848,"kb_used_data":471400,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93561992,"statfs":{"total":96636764160,"available":95807479808,"internally_reserved":0,"allocated":482713600,"data_stored":482454669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481157120,"data_stored":481153641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327692288,"data_stored":327622115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":403787776,"data_stored":403780225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:23.181 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:24.086 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:24.087 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:24.138 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-11T05:18:23.513218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565023380,"num_objects":137949,"num_object_clones":0,"num_object_copies":275898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":137949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143750,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5700,"num_bytes_recovered":23326739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160687,"ondisk_log_size":160687,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2030572,"kb_used_data":1181676,"kb_used_omap":12,"kb_used_meta":848819,"kb_avail":281084948,"statfs":{"total":289910292480,"available":287830986752,"internally_reserved":0,"allocated":1210036224,"data_stored":1209192742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":869190657},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,4,4,4,7,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18325504,"num_objects":-4474,"num_object_clones":0,"num_object_copies":-8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-180,"num_objects_unfound":0,"num_objects_dirty":-4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":737280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013471"},"pg_stats":[{"pgid":"2.f","version":"45'11467","reported_seq":11544,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.166961+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:21.166961+0000","last_peered":"2024-09-11T05:18:21.166961+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:21.166961+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5048},{"shards":"0,2","objects":2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11306","reported_seq":11409,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.354471+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:21.354471+0000","last_peered":"2024-09-11T05:18:21.354471+0000","last_clean":"2024-09-11T05:18:21.354471+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:21.354471+0000","last_undegraded":"2024-09-11T05:18:21.354471+0000","last_fullsized":"2024-09-11T05:18:21.354471+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35201024,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9704,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11139","reported_seq":11266,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.538635+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:21.538635+0000","last_peered":"2024-09-11T05:18:21.538635+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:21.538635+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35135512,"num_objects":8579,"num_object_clones":0,"num_object_copies":17158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9542,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2216},{"shards":"2","objects":5085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11351","reported_seq":11423,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.533467+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:21.533467+0000","last_peered":"2024-09-11T05:18:21.533467+0000","last_clean":"2024-09-11T05:18:21.533467+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:21.533467+0000","last_undegraded":"2024-09-11T05:18:21.533467+0000","last_fullsized":"2024-09-11T05:18:21.533467+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11351,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11425","reported_seq":11555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.115454+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:21.115454+0000","last_peered":"2024-09-11T05:18:21.115454+0000","last_clean":"2024-09-11T05:18:21.115454+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:21.115454+0000","last_undegraded":"2024-09-11T05:18:21.115454+0000","last_fullsized":"2024-09-11T05:18:21.115454+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8259,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11380","reported_seq":15966,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.136842+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:21.136842+0000","last_peered":"2024-09-11T05:18:21.136842+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:21.136842+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2218,"num_bytes_recovered":9076736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7271},{"shards":"0,2","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11463","reported_seq":11596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.115811+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:21.115811+0000","last_peered":"2024-09-11T05:18:21.115811+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:21.115811+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35794944,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5163},{"shards":"0,2","objects":2217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11298","reported_seq":11422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.045747+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:21.045747+0000","last_peered":"2024-09-11T05:18:21.045747+0000","last_clean":"2024-09-11T05:18:21.045747+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:21.045747+0000","last_undegraded":"2024-09-11T05:18:21.045747+0000","last_fullsized":"2024-09-11T05:18:21.045747+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8601,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11336","reported_seq":11468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.569722+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:21.569722+0000","last_peered":"2024-09-11T05:18:21.569722+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:21.569722+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35368960,"num_objects":8635,"num_object_clones":0,"num_object_copies":17270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9404,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2178},{"shards":"2","objects":5110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11132","reported_seq":11298,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.111917+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:21.111917+0000","last_peered":"2024-09-11T05:18:21.111917+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:21.111917+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34979840,"num_objects":8540,"num_object_clones":0,"num_object_copies":17080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7885,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5098},{"shards":"0,2","objects":2150}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11301","reported_seq":11418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.536459+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:21.536459+0000","last_peered":"2024-09-11T05:18:21.536459+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:21.536459+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9330,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2240},{"shards":"2","objects":5210}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11139","reported_seq":18225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.538762+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:21.538762+0000","last_peered":"2024-09-11T05:18:21.538762+0000","last_clean":"2024-09-11T05:18:21.538762+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:21.538762+0000","last_undegraded":"2024-09-11T05:18:21.538762+0000","last_fullsized":"2024-09-11T05:18:21.538762+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098624,"num_objects":8569,"num_object_clones":0,"num_object_copies":17138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11463","reported_seq":11595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.135722+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:21.135722+0000","last_peered":"2024-09-11T05:18:21.135722+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:21.135722+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35557376,"num_objects":8681,"num_object_clones":0,"num_object_copies":17362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8496,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11117","reported_seq":11250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.074346+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:21.074346+0000","last_peered":"2024-09-11T05:18:21.074346+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:21.074346+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34967552,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7983,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11224","reported_seq":11355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.165066+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:21.165066+0000","last_peered":"2024-09-11T05:18:21.165066+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:21.165066+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7982,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11143","reported_seq":11287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.138219+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:21.138219+0000","last_peered":"2024-09-11T05:18:21.138219+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:21.138219+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34599529,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4933},{"shards":"0,2","objects":2173}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565023361,"num_objects":137947,"num_object_clones":0,"num_object_copies":275894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":137947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143748,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5699,"num_bytes_recovered":23326720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1207590912,"data_stored":1207509709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160684,"ondisk_log_size":160684,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638708,"kb_used_data":390428,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93733132,"statfs":{"total":96636764160,"available":95982727168,"internally_reserved":0,"allocated":399798272,"data_stored":399539346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582016,"kb_used_data":319848,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93789824,"statfs":{"total":96636764160,"available":96040779776,"internally_reserved":0,"allocated":327524352,"data_stored":327198727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268431019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215174,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809848,"kb_used_data":471400,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93561992,"statfs":{"total":96636764160,"available":95807479808,"internally_reserved":0,"allocated":482713600,"data_stored":482454669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481157120,"data_stored":481153641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327122944,"data_stored":327052771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399310848,"data_stored":399303297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:24.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:24.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11467", 'reported_seq': 11544, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.166961+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:21.166961+0000', 'last_peered': '2024-09-11T05:18:21.166961+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:21.166961+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35729408, 'num_objects': 8723, 'num_object_clones': 0, 'num_object_copies': 17446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11467, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5048}, {'shards': '0,2', 'objects': 2302}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:24.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11139", 'reported_seq': 11266, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.538635+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:21.538635+0000', 'last_peered': '2024-09-11T05:18:21.538635+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:21.538635+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35135512, 'num_objects': 8579, 'num_object_clones': 0, 'num_object_copies': 17158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8579, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9542, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2216}, {'shards': '2', 'objects': 5085}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:24.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:24.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11380", 'reported_seq': 15966, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.136842+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:21.136842+0000', 'last_peered': '2024-09-11T05:18:21.136842+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:21.136842+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35450880, 'num_objects': 8655, 'num_object_clones': 0, 'num_object_copies': 17310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1382, 'num_objects_unfound': 0, 'num_objects_dirty': 8655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7924, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2218, 'num_bytes_recovered': 9076736, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7271}, {'shards': '0,2', 'objects': 1382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11463", 'reported_seq': 11596, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.115811+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:21.115811+0000', 'last_peered': '2024-09-11T05:18:21.115811+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:21.115811+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35794944, 'num_objects': 8739, 'num_object_clones': 0, 'num_object_copies': 17478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8628, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5163}, {'shards': '0,2', 'objects': 2217}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11336", 'reported_seq': 11468, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.569722+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:21.569722+0000', 'last_peered': '2024-09-11T05:18:21.569722+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:21.569722+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35368960, 'num_objects': 8635, 'num_object_clones': 0, 'num_object_copies': 17270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9404, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2178}, {'shards': '2', 'objects': 5110}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11132", 'reported_seq': 11298, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.111917+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:21.111917+0000', 'last_peered': '2024-09-11T05:18:21.111917+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:21.111917+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34979840, 'num_objects': 8540, 'num_object_clones': 0, 'num_object_copies': 17080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8540, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7885, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5098}, {'shards': '0,2', 'objects': 2150}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11301", 'reported_seq': 11418, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.536459+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:21.536459+0000', 'last_peered': '2024-09-11T05:18:21.536459+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:21.536459+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35766272, 'num_objects': 8732, 'num_object_clones': 0, 'num_object_copies': 17464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9330, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2240}, {'shards': '2', 'objects': 5210}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:24.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11463", 'reported_seq': 11595, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.135722+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:21.135722+0000', 'last_peered': '2024-09-11T05:18:21.135722+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:21.135722+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35557376, 'num_objects': 8681, 'num_object_clones': 0, 'num_object_copies': 17362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8496, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5058}, {'shards': '0,2', 'objects': 2233}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:24.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11117", 'reported_seq': 11250, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.074346+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:21.074346+0000', 'last_peered': '2024-09-11T05:18:21.074346+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:21.074346+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34967552, 'num_objects': 8537, 'num_object_clones': 0, 'num_object_copies': 17074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7983, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5042}, {'shards': '0,2', 'objects': 2209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:24.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11224", 'reported_seq': 11355, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.165066+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:21.165066+0000', 'last_peered': '2024-09-11T05:18:21.165066+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:21.165066+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35127296, 'num_objects': 8576, 'num_object_clones': 0, 'num_object_copies': 17152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7982, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5032}, {'shards': '0,2', 'objects': 2221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:24.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11143", 'reported_seq': 11287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:21.138219+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:21.138219+0000', 'last_peered': '2024-09-11T05:18:21.138219+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:21.138219+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34599529, 'num_objects': 8448, 'num_object_clones': 0, 'num_object_copies': 16896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8150, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4933}, {'shards': '0,2', 'objects': 2173}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:24.142 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:25.309 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:25.309 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/275898 objects degraded (12.842%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":383,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137949,"data_bytes":565023380,"bytes_used":2079305728,"bytes_avail":287830986752,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":275898,"degraded_ratio":0.12842064821057056,"misplaced_objects":1382,"misplaced_total":275898,"misplaced_ratio":0.0050090975650421534,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":61371,"recovering_keys_per_sec":0,"num_objects_recovered":180,"num_bytes_recovered":737280,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:25.351 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:25.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:26.222 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:26.223 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:26.327 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-11T05:18:25.513719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565023380,"num_objects":137949,"num_object_clones":0,"num_object_copies":275898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":137949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143750,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5700,"num_bytes_recovered":23326739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160687,"ondisk_log_size":160687,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2030572,"kb_used_data":1181676,"kb_used_omap":12,"kb_used_meta":848819,"kb_avail":281084948,"statfs":{"total":289910292480,"available":287830986752,"internally_reserved":0,"allocated":1210036224,"data_stored":1209192742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":869190657},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,4,4,4,7,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18325504,"num_objects":-4474,"num_object_clones":0,"num_object_copies":-8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-180,"num_objects_unfound":0,"num_objects_dirty":-4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":737280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013125"},"pg_stats":[{"pgid":"2.f","version":"45'11467","reported_seq":11544,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.166961+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:21.166961+0000","last_peered":"2024-09-11T05:18:21.166961+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:21.166961+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5048},{"shards":"0,2","objects":2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11306","reported_seq":11409,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.354471+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:21.354471+0000","last_peered":"2024-09-11T05:18:21.354471+0000","last_clean":"2024-09-11T05:18:21.354471+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:21.354471+0000","last_undegraded":"2024-09-11T05:18:21.354471+0000","last_fullsized":"2024-09-11T05:18:21.354471+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35201024,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9704,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11139","reported_seq":11266,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.538635+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:21.538635+0000","last_peered":"2024-09-11T05:18:21.538635+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:21.538635+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35135512,"num_objects":8579,"num_object_clones":0,"num_object_copies":17158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9542,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2216},{"shards":"2","objects":5085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11351","reported_seq":11423,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.533467+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:21.533467+0000","last_peered":"2024-09-11T05:18:21.533467+0000","last_clean":"2024-09-11T05:18:21.533467+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:21.533467+0000","last_undegraded":"2024-09-11T05:18:21.533467+0000","last_fullsized":"2024-09-11T05:18:21.533467+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11351,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11425","reported_seq":11555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.115454+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:21.115454+0000","last_peered":"2024-09-11T05:18:21.115454+0000","last_clean":"2024-09-11T05:18:21.115454+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:21.115454+0000","last_undegraded":"2024-09-11T05:18:21.115454+0000","last_fullsized":"2024-09-11T05:18:21.115454+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8259,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11380","reported_seq":15966,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.136842+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:21.136842+0000","last_peered":"2024-09-11T05:18:21.136842+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:21.136842+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7924,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2218,"num_bytes_recovered":9076736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7271},{"shards":"0,2","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11463","reported_seq":11596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.115811+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:21.115811+0000","last_peered":"2024-09-11T05:18:21.115811+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:21.115811+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35794944,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5163},{"shards":"0,2","objects":2217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11298","reported_seq":11422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.045747+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:21.045747+0000","last_peered":"2024-09-11T05:18:21.045747+0000","last_clean":"2024-09-11T05:18:21.045747+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:21.045747+0000","last_undegraded":"2024-09-11T05:18:21.045747+0000","last_fullsized":"2024-09-11T05:18:21.045747+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8601,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11336","reported_seq":11468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.569722+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:21.569722+0000","last_peered":"2024-09-11T05:18:21.569722+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:21.569722+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35368960,"num_objects":8635,"num_object_clones":0,"num_object_copies":17270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9404,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2178},{"shards":"2","objects":5110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11132","reported_seq":11298,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.111917+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:21.111917+0000","last_peered":"2024-09-11T05:18:21.111917+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:21.111917+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34979840,"num_objects":8540,"num_object_clones":0,"num_object_copies":17080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7885,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5098},{"shards":"0,2","objects":2150}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11301","reported_seq":11418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.536459+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:21.536459+0000","last_peered":"2024-09-11T05:18:21.536459+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:21.536459+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9330,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2240},{"shards":"2","objects":5210}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11139","reported_seq":18225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:21.538762+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:21.538762+0000","last_peered":"2024-09-11T05:18:21.538762+0000","last_clean":"2024-09-11T05:18:21.538762+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:21.538762+0000","last_undegraded":"2024-09-11T05:18:21.538762+0000","last_fullsized":"2024-09-11T05:18:21.538762+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098624,"num_objects":8569,"num_object_clones":0,"num_object_copies":17138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11463","reported_seq":11595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.135722+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:21.135722+0000","last_peered":"2024-09-11T05:18:21.135722+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:21.135722+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35557376,"num_objects":8681,"num_object_clones":0,"num_object_copies":17362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8496,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11117","reported_seq":11250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.074346+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:21.074346+0000","last_peered":"2024-09-11T05:18:21.074346+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:21.074346+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34967552,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7983,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5042},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11224","reported_seq":11355,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.165066+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:21.165066+0000","last_peered":"2024-09-11T05:18:21.165066+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:21.165066+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7982,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11143","reported_seq":11287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:21.138219+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:21.138219+0000","last_peered":"2024-09-11T05:18:21.138219+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:21.138219+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34599529,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8150,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4933},{"shards":"0,2","objects":2173}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565023361,"num_objects":137947,"num_object_clones":0,"num_object_copies":275894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1382,"num_objects_unfound":0,"num_objects_dirty":137947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143748,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5699,"num_bytes_recovered":23326720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1207590912,"data_stored":1207509709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160684,"ondisk_log_size":160684,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638708,"kb_used_data":390428,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93733132,"statfs":{"total":96636764160,"available":95982727168,"internally_reserved":0,"allocated":399798272,"data_stored":399539346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":254209707},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582016,"kb_used_data":319848,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93789824,"statfs":{"total":96636764160,"available":96040779776,"internally_reserved":0,"allocated":327524352,"data_stored":327198727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268431019},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215174,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809848,"kb_used_data":471400,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93561992,"statfs":{"total":96636764160,"available":95807479808,"internally_reserved":0,"allocated":482713600,"data_stored":482454669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481157120,"data_stored":481153641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327122944,"data_stored":327052771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399310848,"data_stored":399303297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:29.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:30.137 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:30.139 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:30.186 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-11T05:18:29.514736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555987604,"num_objects":135743,"num_object_clones":0,"num_object_copies":271486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":135743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145956,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5803,"num_bytes_recovered":23748627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2075540,"kb_used_data":1169752,"kb_used_omap":12,"kb_used_meta":905715,"kb_avail":281039980,"statfs":{"total":289910292480,"available":287784939520,"internally_reserved":0,"allocated":1197826048,"data_stored":1196982566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":927452161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,5,7,13,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18079744,"num_objects":-4414,"num_object_clones":0,"num_object_copies":-8828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-197,"num_objects_unfound":0,"num_objects_dirty":-4414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4414,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012292"},"pg_stats":[{"pgid":"2.f","version":"45'11615","reported_seq":11694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.119225+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:26.119225+0000","last_peered":"2024-09-11T05:18:26.119225+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:26.119225+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35123200,"num_objects":8575,"num_object_clones":0,"num_object_copies":17150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11615,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4900},{"shards":"0,2","objects":2154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11464","reported_seq":11567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.613711+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:26.613711+0000","last_peered":"2024-09-11T05:18:26.613711+0000","last_clean":"2024-09-11T05:18:26.613711+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:26.613711+0000","last_undegraded":"2024-09-11T05:18:26.613711+0000","last_fullsized":"2024-09-11T05:18:26.613711+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34553856,"num_objects":8436,"num_object_clones":0,"num_object_copies":16872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9862,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11285","reported_seq":11412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.594022+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:26.594022+0000","last_peered":"2024-09-11T05:18:26.594022+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:26.594022+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537496,"num_objects":8433,"num_object_clones":0,"num_object_copies":16866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9688,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2070},{"shards":"2","objects":4939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11484","reported_seq":11556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.555201+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:26.555201+0000","last_peered":"2024-09-11T05:18:26.555201+0000","last_clean":"2024-09-11T05:18:26.555201+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:26.555201+0000","last_undegraded":"2024-09-11T05:18:26.555201+0000","last_fullsized":"2024-09-11T05:18:26.555201+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34717696,"num_objects":8476,"num_object_clones":0,"num_object_copies":16952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11484,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11561","reported_seq":11691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.177930+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:26.177930+0000","last_peered":"2024-09-11T05:18:26.177930+0000","last_clean":"2024-09-11T05:18:26.177930+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:26.177930+0000","last_undegraded":"2024-09-11T05:18:26.177930+0000","last_fullsized":"2024-09-11T05:18:26.177930+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258368,"num_objects":8608,"num_object_clones":0,"num_object_copies":17216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11523","reported_seq":16314,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.185576+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:26.185576+0000","last_peered":"2024-09-11T05:18:26.185576+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:26.185576+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34865152,"num_objects":8512,"num_object_clones":0,"num_object_copies":17024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":8512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8067,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2321,"num_bytes_recovered":9498624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7231},{"shards":"0,2","objects":1279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11609","reported_seq":11740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.128816+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:26.128816+0000","last_peered":"2024-09-11T05:18:26.128816+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:26.128816+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8774,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5017},{"shards":"0,2","objects":2071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11442","reported_seq":11567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.056209+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:26.056209+0000","last_peered":"2024-09-11T05:18:26.056209+0000","last_clean":"2024-09-11T05:18:26.056209+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:26.056209+0000","last_undegraded":"2024-09-11T05:18:26.056209+0000","last_fullsized":"2024-09-11T05:18:26.056209+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11457","reported_seq":11589,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.429839+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:26.429839+0000","last_peered":"2024-09-11T05:18:26.429839+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:26.429839+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2057},{"shards":"2","objects":4989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11257","reported_seq":11422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.187999+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:26.187999+0000","last_peered":"2024-09-11T05:18:26.187999+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:26.187999+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8010,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4973},{"shards":"0,2","objects":2025}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11442","reported_seq":11560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.596490+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:26.596490+0000","last_peered":"2024-09-11T05:18:26.596490+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:26.596490+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2099},{"shards":"2","objects":5069}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11267","reported_seq":18353,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.448243+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:26.448243+0000","last_peered":"2024-09-11T05:18:26.448243+0000","last_clean":"2024-09-11T05:18:26.448243+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:26.448243+0000","last_undegraded":"2024-09-11T05:18:26.448243+0000","last_fullsized":"2024-09-11T05:18:26.448243+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34574336,"num_objects":8441,"num_object_clones":0,"num_object_copies":16882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9170,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11595","reported_seq":11727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.086399+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:26.086399+0000","last_peered":"2024-09-11T05:18:26.086399+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:26.086399+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35016704,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4926},{"shards":"0,2","objects":2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11240","reported_seq":11374,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.194601+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:26.194601+0000","last_peered":"2024-09-11T05:18:26.194601+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:26.194601+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8106,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":2086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11368","reported_seq":11504,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.095974+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:26.095974+0000","last_peered":"2024-09-11T05:18:26.095974+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:26.095974+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537472,"num_objects":8432,"num_object_clones":0,"num_object_copies":16864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8126,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4888},{"shards":"0,2","objects":2077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11281","reported_seq":11425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.084058+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:26.084058+0000","last_peered":"2024-09-11T05:18:26.084058+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:26.084058+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34034281,"num_objects":8310,"num_object_clones":0,"num_object_copies":16620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4795},{"shards":"0,2","objects":2035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":555987585,"num_objects":135741,"num_object_clones":0,"num_object_copies":271482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":135741,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145954,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5802,"num_bytes_recovered":23748608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1195376640,"data_stored":1195295437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656468,"kb_used_data":385988,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93715372,"statfs":{"total":96636764160,"available":95964540928,"internally_reserved":0,"allocated":395251712,"data_stored":394992786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607588,"kb_used_data":319056,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93764252,"statfs":{"total":96636764160,"available":96014594048,"internally_reserved":0,"allocated":326713344,"data_stored":326387719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215175,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":811484,"kb_used_data":464708,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93560356,"statfs":{"total":96636764160,"available":95805804544,"internally_reserved":0,"allocated":475860992,"data_stored":475602061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,6,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474300416,"data_stored":474296937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326316032,"data_stored":326245859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":394760192,"data_stored":394752641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:30.188 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:31.065 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:31.065 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:31.177 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-11T05:18:29.514736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555987604,"num_objects":135743,"num_object_clones":0,"num_object_copies":271486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":135743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145956,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5803,"num_bytes_recovered":23748627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2075540,"kb_used_data":1169752,"kb_used_omap":12,"kb_used_meta":905715,"kb_avail":281039980,"statfs":{"total":289910292480,"available":287784939520,"internally_reserved":0,"allocated":1197826048,"data_stored":1196982566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":927452161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,5,7,13,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18079744,"num_objects":-4414,"num_object_clones":0,"num_object_copies":-8828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-197,"num_objects_unfound":0,"num_objects_dirty":-4414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4414,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012292"},"pg_stats":[{"pgid":"2.f","version":"45'11615","reported_seq":11694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.119225+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:26.119225+0000","last_peered":"2024-09-11T05:18:26.119225+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:26.119225+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35123200,"num_objects":8575,"num_object_clones":0,"num_object_copies":17150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11615,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4900},{"shards":"0,2","objects":2154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11464","reported_seq":11567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.613711+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:26.613711+0000","last_peered":"2024-09-11T05:18:26.613711+0000","last_clean":"2024-09-11T05:18:26.613711+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:26.613711+0000","last_undegraded":"2024-09-11T05:18:26.613711+0000","last_fullsized":"2024-09-11T05:18:26.613711+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34553856,"num_objects":8436,"num_object_clones":0,"num_object_copies":16872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9862,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11285","reported_seq":11412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.594022+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:26.594022+0000","last_peered":"2024-09-11T05:18:26.594022+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:26.594022+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537496,"num_objects":8433,"num_object_clones":0,"num_object_copies":16866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9688,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2070},{"shards":"2","objects":4939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11484","reported_seq":11556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.555201+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:26.555201+0000","last_peered":"2024-09-11T05:18:26.555201+0000","last_clean":"2024-09-11T05:18:26.555201+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:26.555201+0000","last_undegraded":"2024-09-11T05:18:26.555201+0000","last_fullsized":"2024-09-11T05:18:26.555201+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34717696,"num_objects":8476,"num_object_clones":0,"num_object_copies":16952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11484,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11561","reported_seq":11691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.177930+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:26.177930+0000","last_peered":"2024-09-11T05:18:26.177930+0000","last_clean":"2024-09-11T05:18:26.177930+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:26.177930+0000","last_undegraded":"2024-09-11T05:18:26.177930+0000","last_fullsized":"2024-09-11T05:18:26.177930+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258368,"num_objects":8608,"num_object_clones":0,"num_object_copies":17216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11523","reported_seq":16314,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.185576+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:26.185576+0000","last_peered":"2024-09-11T05:18:26.185576+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:26.185576+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34865152,"num_objects":8512,"num_object_clones":0,"num_object_copies":17024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":8512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8067,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2321,"num_bytes_recovered":9498624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7231},{"shards":"0,2","objects":1279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11609","reported_seq":11740,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.128816+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:26.128816+0000","last_peered":"2024-09-11T05:18:26.128816+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:26.128816+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8774,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5017},{"shards":"0,2","objects":2071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11442","reported_seq":11567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.056209+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:26.056209+0000","last_peered":"2024-09-11T05:18:26.056209+0000","last_clean":"2024-09-11T05:18:26.056209+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:26.056209+0000","last_undegraded":"2024-09-11T05:18:26.056209+0000","last_fullsized":"2024-09-11T05:18:26.056209+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11457","reported_seq":11589,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.429839+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:26.429839+0000","last_peered":"2024-09-11T05:18:26.429839+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:26.429839+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2057},{"shards":"2","objects":4989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11257","reported_seq":11422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.187999+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:26.187999+0000","last_peered":"2024-09-11T05:18:26.187999+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:26.187999+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8010,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4973},{"shards":"0,2","objects":2025}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11442","reported_seq":11560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.596490+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:26.596490+0000","last_peered":"2024-09-11T05:18:26.596490+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:26.596490+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2099},{"shards":"2","objects":5069}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11267","reported_seq":18353,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.448243+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:26.448243+0000","last_peered":"2024-09-11T05:18:26.448243+0000","last_clean":"2024-09-11T05:18:26.448243+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:26.448243+0000","last_undegraded":"2024-09-11T05:18:26.448243+0000","last_fullsized":"2024-09-11T05:18:26.448243+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34574336,"num_objects":8441,"num_object_clones":0,"num_object_copies":16882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9170,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11595","reported_seq":11727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.086399+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:26.086399+0000","last_peered":"2024-09-11T05:18:26.086399+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:26.086399+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35016704,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4926},{"shards":"0,2","objects":2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11240","reported_seq":11374,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.194601+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:26.194601+0000","last_peered":"2024-09-11T05:18:26.194601+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:26.194601+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8106,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":2086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11368","reported_seq":11504,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.095974+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:26.095974+0000","last_peered":"2024-09-11T05:18:26.095974+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:26.095974+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537472,"num_objects":8432,"num_object_clones":0,"num_object_copies":16864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8126,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4888},{"shards":"0,2","objects":2077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11281","reported_seq":11425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.084058+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:26.084058+0000","last_peered":"2024-09-11T05:18:26.084058+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:26.084058+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34034281,"num_objects":8310,"num_object_clones":0,"num_object_copies":16620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4795},{"shards":"0,2","objects":2035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":555987585,"num_objects":135741,"num_object_clones":0,"num_object_copies":271482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1279,"num_objects_unfound":0,"num_objects_dirty":135741,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145954,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5802,"num_bytes_recovered":23748608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1195376640,"data_stored":1195295437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656468,"kb_used_data":385988,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93715372,"statfs":{"total":96636764160,"available":95964540928,"internally_reserved":0,"allocated":395251712,"data_stored":394992786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607588,"kb_used_data":319056,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93764252,"statfs":{"total":96636764160,"available":96014594048,"internally_reserved":0,"allocated":326713344,"data_stored":326387719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215175,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":811484,"kb_used_data":464708,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93560356,"statfs":{"total":96636764160,"available":95805804544,"internally_reserved":0,"allocated":475860992,"data_stored":475602061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,6,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474300416,"data_stored":474296937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326316032,"data_stored":326245859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":394760192,"data_stored":394752641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11615", 'reported_seq': 11694, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.119225+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:26.119225+0000', 'last_peered': '2024-09-11T05:18:26.119225+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:26.119225+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35123200, 'num_objects': 8575, 'num_object_clones': 0, 'num_object_copies': 17150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11615, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4900}, {'shards': '0,2', 'objects': 2154}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11285", 'reported_seq': 11412, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.594022+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:26.594022+0000', 'last_peered': '2024-09-11T05:18:26.594022+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:26.594022+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34537496, 'num_objects': 8433, 'num_object_clones': 0, 'num_object_copies': 16866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8433, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9688, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2070}, {'shards': '2', 'objects': 4939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11523", 'reported_seq': 16314, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.185576+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:26.185576+0000', 'last_peered': '2024-09-11T05:18:26.185576+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:26.185576+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34865152, 'num_objects': 8512, 'num_object_clones': 0, 'num_object_copies': 17024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1279, 'num_objects_unfound': 0, 'num_objects_dirty': 8512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8067, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2321, 'num_bytes_recovered': 9498624, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7231}, {'shards': '0,2', 'objects': 1279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11609", 'reported_seq': 11740, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.128816+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:26.128816+0000', 'last_peered': '2024-09-11T05:18:26.128816+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:26.128816+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35196928, 'num_objects': 8593, 'num_object_clones': 0, 'num_object_copies': 17186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8774, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5017}, {'shards': '0,2', 'objects': 2071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:31.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11457", 'reported_seq': 11589, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.429839+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:26.429839+0000', 'last_peered': '2024-09-11T05:18:26.429839+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:26.429839+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34873344, 'num_objects': 8514, 'num_object_clones': 0, 'num_object_copies': 17028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9525, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2057}, {'shards': '2', 'objects': 4989}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:31.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:31.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11257", 'reported_seq': 11422, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.187999+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:26.187999+0000', 'last_peered': '2024-09-11T05:18:26.187999+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:26.187999+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34467840, 'num_objects': 8415, 'num_object_clones': 0, 'num_object_copies': 16830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8010, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4973}, {'shards': '0,2', 'objects': 2025}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:31.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11442", 'reported_seq': 11560, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.596490+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:26.596490+0000', 'last_peered': '2024-09-11T05:18:26.596490+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:26.596490+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35188736, 'num_objects': 8591, 'num_object_clones': 0, 'num_object_copies': 17182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9471, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2099}, {'shards': '2', 'objects': 5069}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:31.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:31.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11595", 'reported_seq': 11727, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.086399+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:26.086399+0000', 'last_peered': '2024-09-11T05:18:26.086399+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:26.086399+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35016704, 'num_objects': 8549, 'num_object_clones': 0, 'num_object_copies': 17098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8628, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4926}, {'shards': '0,2', 'objects': 2101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:31.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11240", 'reported_seq': 11374, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.194601+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:26.194601+0000', 'last_peered': '2024-09-11T05:18:26.194601+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:26.194601+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34463744, 'num_objects': 8414, 'num_object_clones': 0, 'num_object_copies': 16828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8106, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4919}, {'shards': '0,2', 'objects': 2086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:31.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11368", 'reported_seq': 11504, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.095974+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:26.095974+0000', 'last_peered': '2024-09-11T05:18:26.095974+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:26.095974+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34537472, 'num_objects': 8432, 'num_object_clones': 0, 'num_object_copies': 16864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8126, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4888}, {'shards': '0,2', 'objects': 2077}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:31.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11281", 'reported_seq': 11425, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:26.084058+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:26.084058+0000', 'last_peered': '2024-09-11T05:18:26.084058+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:26.084058+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34034281, 'num_objects': 8310, 'num_object_clones': 0, 'num_object_copies': 16620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8288, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4795}, {'shards': '0,2', 'objects': 2035}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:31.182 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:32.289 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:32.289 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/268682 objects degraded (13.187%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":390,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134341,"data_bytes":550245012,"bytes_used":2124120064,"bytes_avail":287786172416,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":268682,"degraded_ratio":0.13186964515672803,"misplaced_objects":1192,"misplaced_total":268682,"misplaced_ratio":0.0044364713676390676,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96914,"recovering_keys_per_sec":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:32.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:32.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:33.000 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:33.000 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:33.058 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-11T05:18:31.515179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":550245012,"num_objects":134341,"num_object_clones":0,"num_object_copies":268682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1192,"num_objects_unfound":0,"num_objects_dirty":134341,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147358,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5890,"num_bytes_recovered":24104979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2074336,"kb_used_data":1162988,"kb_used_omap":12,"kb_used_meta":911283,"kb_avail":281041184,"statfs":{"total":289910292480,"available":287786172416,"internally_reserved":0,"allocated":1190899712,"data_stored":1190056230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":933153793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,4,8,15,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23822336,"num_objects":-5816,"num_object_clones":0,"num_object_copies":-11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":-5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"2.f","version":"45'11759","reported_seq":11838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.187447+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:31.187447+0000","last_peered":"2024-09-11T05:18:31.187447+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:31.187447+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11759,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4756},{"shards":"0,2","objects":2010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11464","reported_seq":11567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.613711+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:26.613711+0000","last_peered":"2024-09-11T05:18:26.613711+0000","last_clean":"2024-09-11T05:18:26.613711+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:26.613711+0000","last_undegraded":"2024-09-11T05:18:26.613711+0000","last_fullsized":"2024-09-11T05:18:26.613711+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34553856,"num_objects":8436,"num_object_clones":0,"num_object_copies":16872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9862,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11285","reported_seq":11412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.594022+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:26.594022+0000","last_peered":"2024-09-11T05:18:26.594022+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:26.594022+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34537496,"num_objects":8433,"num_object_clones":0,"num_object_copies":16866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9688,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2070},{"shards":"2","objects":4939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11484","reported_seq":11556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.555201+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:26.555201+0000","last_peered":"2024-09-11T05:18:26.555201+0000","last_clean":"2024-09-11T05:18:26.555201+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:26.555201+0000","last_undegraded":"2024-09-11T05:18:26.555201+0000","last_fullsized":"2024-09-11T05:18:26.555201+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34717696,"num_objects":8476,"num_object_clones":0,"num_object_copies":16952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11484,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11714","reported_seq":11844,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:31.111850+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:31.111850+0000","last_peered":"2024-09-11T05:18:31.111850+0000","last_clean":"2024-09-11T05:18:31.111850+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:31.111850+0000","last_undegraded":"2024-09-11T05:18:31.111850+0000","last_fullsized":"2024-09-11T05:18:31.111850+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34631680,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8548,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11682","reported_seq":16647,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.202355+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:31.202355+0000","last_peered":"2024-09-11T05:18:31.202355+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:31.202355+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34213888,"num_objects":8353,"num_object_clones":0,"num_object_copies":16706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1192,"num_objects_unfound":0,"num_objects_dirty":8353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8226,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2408,"num_bytes_recovered":9854976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7159},{"shards":"0,2","objects":1192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11741","reported_seq":11874,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.110257+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:31.110257+0000","last_peered":"2024-09-11T05:18:31.110257+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:31.110257+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34656256,"num_objects":8461,"num_object_clones":0,"num_object_copies":16922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4885},{"shards":"0,2","objects":1939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11561","reported_seq":11686,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:30.922351+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:30.922351+0000","last_peered":"2024-09-11T05:18:30.922351+0000","last_clean":"2024-09-11T05:18:30.922351+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:30.922351+0000","last_undegraded":"2024-09-11T05:18:30.922351+0000","last_fullsized":"2024-09-11T05:18:30.922351+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34091008,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8864,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11457","reported_seq":11589,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.429839+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:26.429839+0000","last_peered":"2024-09-11T05:18:26.429839+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:26.429839+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2057},{"shards":"2","objects":4989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11382","reported_seq":11548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.163249+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:31.163249+0000","last_peered":"2024-09-11T05:18:31.163249+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:31.163249+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33955840,"num_objects":8290,"num_object_clones":0,"num_object_copies":16580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8135,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4848},{"shards":"0,2","objects":1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11442","reported_seq":11560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:26.596490+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:26.596490+0000","last_peered":"2024-09-11T05:18:26.596490+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:26.596490+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2099},{"shards":"2","objects":5069}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11267","reported_seq":18353,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:26.448243+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:26.448243+0000","last_peered":"2024-09-11T05:18:26.448243+0000","last_clean":"2024-09-11T05:18:26.448243+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:26.448243+0000","last_undegraded":"2024-09-11T05:18:26.448243+0000","last_fullsized":"2024-09-11T05:18:26.448243+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34574336,"num_objects":8441,"num_object_clones":0,"num_object_copies":16882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9170,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11723","reported_seq":11855,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.152806+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:31.152806+0000","last_peered":"2024-09-11T05:18:31.152806+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:31.152806+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8756,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4798},{"shards":"0,2","objects":1973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11381","reported_seq":11515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.120719+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:31.120719+0000","last_peered":"2024-09-11T05:18:31.120719+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:31.120719+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33886208,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8247,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4778},{"shards":"0,2","objects":1945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11512","reported_seq":11646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.031545+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:31.031545+0000","last_peered":"2024-09-11T05:18:31.031545+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:31.031545+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33947648,"num_objects":8288,"num_object_clones":0,"num_object_copies":16576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8270,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":1933}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.570197+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:15:57.570197+0000","last_peered":"2024-09-11T05:15:57.570197+0000","last_clean":"2024-09-11T05:15:57.570197+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:15:57.570197+0000","last_undegraded":"2024-09-11T05:15:57.570197+0000","last_fullsized":"2024-09-11T05:15:57.570197+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11438","reported_seq":11582,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.201716+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:31.201716+0000","last_peered":"2024-09-11T05:18:31.201716+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:31.201716+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33391209,"num_objects":8153,"num_object_clones":0,"num_object_copies":16306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8445,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4638},{"shards":"0,2","objects":1878}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":550244993,"num_objects":134339,"num_object_clones":0,"num_object_copies":268678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1192,"num_objects_unfound":0,"num_objects_dirty":134339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147356,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5889,"num_bytes_recovered":24104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188474880,"data_stored":1188393677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656468,"kb_used_data":385988,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93715372,"statfs":{"total":96636764160,"available":95964540928,"internally_reserved":0,"allocated":395251712,"data_stored":394992786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276950699},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607588,"kb_used_data":319056,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93764252,"statfs":{"total":96636764160,"available":96014594048,"internally_reserved":0,"allocated":326713344,"data_stored":326387719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215176,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810280,"kb_used_data":457944,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93561560,"statfs":{"total":96636764160,"available":95807037440,"internally_reserved":0,"allocated":468934656,"data_stored":468675725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360771243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,2,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467398656,"data_stored":467395177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326316032,"data_stored":326245859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":394760192,"data_stored":394752641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:36.060 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:36.893 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:36.894 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:36.933 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-11T05:18:35.519078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546689684,"num_objects":133473,"num_object_clones":0,"num_object_copies":266946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1192,"num_objects_unfound":0,"num_objects_dirty":133473,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148226,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5890,"num_bytes_recovered":24104979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2077296,"kb_used_data":1157624,"kb_used_omap":12,"kb_used_meta":919603,"kb_avail":281038224,"statfs":{"total":289910292480,"available":287783141376,"internally_reserved":0,"allocated":1185406976,"data_stored":1184563494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":941673473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,7,1,5,13,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18333696,"num_objects":-4476,"num_object_clones":0,"num_object_copies":-8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-190,"num_objects_unfound":0,"num_objects_dirty":-4476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4476,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005860"},"pg_stats":[{"pgid":"2.f","version":"45'11759","reported_seq":11838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.187447+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:31.187447+0000","last_peered":"2024-09-11T05:18:31.187447+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:31.187447+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11759,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4756},{"shards":"0,2","objects":2010}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11603","reported_seq":11706,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:31.540401+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:31.540401+0000","last_peered":"2024-09-11T05:18:31.540401+0000","last_clean":"2024-09-11T05:18:31.540401+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:31.540401+0000","last_undegraded":"2024-09-11T05:18:31.540401+0000","last_fullsized":"2024-09-11T05:18:31.540401+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33984512,"num_objects":8297,"num_object_clones":0,"num_object_copies":16594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10001,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11425","reported_seq":11552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.478865+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:31.478865+0000","last_peered":"2024-09-11T05:18:31.478865+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:31.478865+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33964056,"num_objects":8293,"num_object_clones":0,"num_object_copies":16586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8293,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9828,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":4799}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11627","reported_seq":11699,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:31.630952+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:31.630952+0000","last_peered":"2024-09-11T05:18:31.630952+0000","last_clean":"2024-09-11T05:18:31.630952+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:31.630952+0000","last_undegraded":"2024-09-11T05:18:31.630952+0000","last_fullsized":"2024-09-11T05:18:31.630952+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34131968,"num_objects":8333,"num_object_clones":0,"num_object_copies":16666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11627,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11714","reported_seq":11844,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:31.111850+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:31.111850+0000","last_peered":"2024-09-11T05:18:31.111850+0000","last_clean":"2024-09-11T05:18:31.111850+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:31.111850+0000","last_undegraded":"2024-09-11T05:18:31.111850+0000","last_fullsized":"2024-09-11T05:18:31.111850+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34631680,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8548,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11682","reported_seq":16647,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.202355+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:31.202355+0000","last_peered":"2024-09-11T05:18:31.202355+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:31.202355+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34213888,"num_objects":8353,"num_object_clones":0,"num_object_copies":16706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1192,"num_objects_unfound":0,"num_objects_dirty":8353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8226,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2408,"num_bytes_recovered":9854976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7159},{"shards":"0,2","objects":1192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11741","reported_seq":11874,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.110257+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:31.110257+0000","last_peered":"2024-09-11T05:18:31.110257+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:31.110257+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34656256,"num_objects":8461,"num_object_clones":0,"num_object_copies":16922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4885},{"shards":"0,2","objects":1939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11561","reported_seq":11686,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:30.922351+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:30.922351+0000","last_peered":"2024-09-11T05:18:30.922351+0000","last_clean":"2024-09-11T05:18:30.922351+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:30.922351+0000","last_undegraded":"2024-09-11T05:18:30.922351+0000","last_fullsized":"2024-09-11T05:18:30.922351+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34091008,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8864,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11600","reported_seq":11731,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.540767+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:31.540767+0000","last_peered":"2024-09-11T05:18:31.540767+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:31.540767+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34287616,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9668,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1914},{"shards":"2","objects":4846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11382","reported_seq":11548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.163249+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:31.163249+0000","last_peered":"2024-09-11T05:18:31.163249+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:31.163249+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33955840,"num_objects":8290,"num_object_clones":0,"num_object_copies":16580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8135,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4848},{"shards":"0,2","objects":1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11576","reported_seq":11694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.627705+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:31.627705+0000","last_peered":"2024-09-11T05:18:31.627705+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:31.627705+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34639872,"num_objects":8457,"num_object_clones":0,"num_object_copies":16914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9605,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1965},{"shards":"2","objects":4935}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11436","reported_seq":18522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:31.645799+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:31.645799+0000","last_peered":"2024-09-11T05:18:31.645799+0000","last_clean":"2024-09-11T05:18:31.645799+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:31.645799+0000","last_undegraded":"2024-09-11T05:18:31.645799+0000","last_fullsized":"2024-09-11T05:18:31.645799+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33882112,"num_objects":8272,"num_object_clones":0,"num_object_copies":16544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9339,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11723","reported_seq":11855,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.152806+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:31.152806+0000","last_peered":"2024-09-11T05:18:31.152806+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:31.152806+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8756,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4798},{"shards":"0,2","objects":1973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11381","reported_seq":11515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.120719+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:31.120719+0000","last_peered":"2024-09-11T05:18:31.120719+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:31.120719+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33886208,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8247,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4778},{"shards":"0,2","objects":1945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11512","reported_seq":11646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.031545+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:31.031545+0000","last_peered":"2024-09-11T05:18:31.031545+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:31.031545+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33947648,"num_objects":8288,"num_object_clones":0,"num_object_copies":16576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8270,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":1933}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11438","reported_seq":11582,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:31.201716+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:31.201716+0000","last_peered":"2024-09-11T05:18:31.201716+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:31.201716+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33391209,"num_objects":8153,"num_object_clones":0,"num_object_copies":16306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8445,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4638},{"shards":"0,2","objects":1878}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546689665,"num_objects":133471,"num_object_clones":0,"num_object_copies":266942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1192,"num_objects_unfound":0,"num_objects_dirty":133471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148224,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5889,"num_bytes_recovered":24104960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1182818304,"data_stored":1182737101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160760,"ondisk_log_size":160760,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656148,"kb_used_data":381500,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93715692,"statfs":{"total":96636764160,"available":95964868608,"internally_reserved":0,"allocated":390656000,"data_stored":390397074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281210539},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,0,2,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610868,"kb_used_data":318180,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93760972,"statfs":{"total":96636764160,"available":96011235328,"internally_reserved":0,"allocated":325816320,"data_stored":325490695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215176,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810280,"kb_used_data":457944,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93561560,"statfs":{"total":96636764160,"available":95807037440,"internally_reserved":0,"allocated":468934656,"data_stored":468675725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360771243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,2,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467398656,"data_stored":467395177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325266915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390082560,"data_stored":390075009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:36.934 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:37.736 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:37.739 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-11T05:18:37.519511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537383572,"num_objects":131201,"num_object_clones":0,"num_object_copies":262402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1111,"num_objects_unfound":0,"num_objects_dirty":131201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150498,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5971,"num_bytes_recovered":24436755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2092556,"kb_used_data":1146520,"kb_used_omap":12,"kb_used_meta":945971,"kb_avail":281022964,"statfs":{"total":289910292480,"available":287767515136,"internally_reserved":0,"allocated":1174036480,"data_stored":1173192998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":968674305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,6,4,8,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27639808,"num_objects":-6748,"num_object_clones":0,"num_object_copies":-13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005791"},"pg_stats":[{"pgid":"2.f","version":"45'11896","reported_seq":11975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.258176+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:36.258176+0000","last_peered":"2024-09-11T05:18:36.258176+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:36.258176+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33972224,"num_objects":8294,"num_object_clones":0,"num_object_copies":16588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11896,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4619},{"shards":"0,2","objects":1873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11753","reported_seq":11856,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.567704+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:36.567704+0000","last_peered":"2024-09-11T05:18:36.567704+0000","last_clean":"2024-09-11T05:18:36.567704+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:36.567704+0000","last_undegraded":"2024-09-11T05:18:36.567704+0000","last_fullsized":"2024-09-11T05:18:36.567704+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33370112,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10151,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11567","reported_seq":11694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.625783+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:36.625783+0000","last_peered":"2024-09-11T05:18:36.625783+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:36.625783+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382424,"num_objects":8151,"num_object_clones":0,"num_object_copies":16302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9970,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1788},{"shards":"2","objects":4657}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11784","reported_seq":11855,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.601101+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:36.601101+0000","last_peered":"2024-09-11T05:18:36.601101+0000","last_clean":"2024-09-11T05:18:36.601101+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:36.601101+0000","last_undegraded":"2024-09-11T05:18:36.601101+0000","last_fullsized":"2024-09-11T05:18:36.601101+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33488896,"num_objects":8176,"num_object_clones":0,"num_object_copies":16352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11784,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11850","reported_seq":11980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.160346+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:36.160346+0000","last_peered":"2024-09-11T05:18:36.160346+0000","last_clean":"2024-09-11T05:18:36.160346+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:36.160346+0000","last_undegraded":"2024-09-11T05:18:36.160346+0000","last_fullsized":"2024-09-11T05:18:36.160346+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34074624,"num_objects":8319,"num_object_clones":0,"num_object_copies":16638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8684,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11852","reported_seq":16980,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.243019+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:36.243019+0000","last_peered":"2024-09-11T05:18:36.243019+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:36.243019+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33517568,"num_objects":8183,"num_object_clones":0,"num_object_copies":16366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1111,"num_objects_unfound":0,"num_objects_dirty":8183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8396,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2489,"num_bytes_recovered":10186752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7070},{"shards":"0,2","objects":1111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11882","reported_seq":12015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.200362+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:36.200362+0000","last_peered":"2024-09-11T05:18:36.200362+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:36.200362+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34078720,"num_objects":8320,"num_object_clones":0,"num_object_copies":16640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9047,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11694","reported_seq":11819,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.186527+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:36.186527+0000","last_peered":"2024-09-11T05:18:36.186527+0000","last_clean":"2024-09-11T05:18:36.186527+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:36.186527+0000","last_undegraded":"2024-09-11T05:18:36.186527+0000","last_fullsized":"2024-09-11T05:18:36.186527+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33546240,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8997,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11733","reported_seq":11865,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.644576+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:36.644576+0000","last_peered":"2024-09-11T05:18:36.644576+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:36.644576+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33742848,"num_objects":8238,"num_object_clones":0,"num_object_copies":16476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1781},{"shards":"2","objects":4713}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11542","reported_seq":11708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.252981+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:36.252981+0000","last_peered":"2024-09-11T05:18:36.252981+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:36.252981+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8295,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4688},{"shards":"0,2","objects":1740}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11719","reported_seq":11837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.506514+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:36.506514+0000","last_peered":"2024-09-11T05:18:36.506514+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:36.506514+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34054144,"num_objects":8314,"num_object_clones":0,"num_object_copies":16628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9748,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1822},{"shards":"2","objects":4792}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11578","reported_seq":18664,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.468989+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:36.468989+0000","last_peered":"2024-09-11T05:18:36.468989+0000","last_clean":"2024-09-11T05:18:36.468989+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:36.468989+0000","last_undegraded":"2024-09-11T05:18:36.468989+0000","last_fullsized":"2024-09-11T05:18:36.468989+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9481,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11860","reported_seq":11992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.128942+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:36.128942+0000","last_peered":"2024-09-11T05:18:36.128942+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:36.128942+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33931264,"num_objects":8284,"num_object_clones":0,"num_object_copies":16568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8893,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4661},{"shards":"0,2","objects":1836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11506","reported_seq":11640,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.187181+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:36.187181+0000","last_peered":"2024-09-11T05:18:36.187181+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:36.187181+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33374208,"num_objects":8148,"num_object_clones":0,"num_object_copies":16296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8372,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4653},{"shards":"0,2","objects":1820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11638","reported_seq":11774,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.185979+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:36.185979+0000","last_peered":"2024-09-11T05:18:36.185979+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:36.185979+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33431552,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8396,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4618},{"shards":"0,2","objects":1807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11578","reported_seq":11722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.250354+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:36.250354+0000","last_peered":"2024-09-11T05:18:36.250354+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:36.250354+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817769,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8585,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4498},{"shards":"0,2","objects":1738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537383553,"num_objects":131199,"num_object_clones":0,"num_object_copies":262398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1111,"num_objects_unfound":0,"num_objects_dirty":131199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150496,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5970,"num_bytes_recovered":24436736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1171156992,"data_stored":1171075789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673916,"kb_used_data":377068,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93697924,"statfs":{"total":96636764160,"available":95946674176,"internally_reserved":0,"allocated":386117632,"data_stored":385858706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610868,"kb_used_data":318180,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93760972,"statfs":{"total":96636764160,"available":96011235328,"internally_reserved":0,"allocated":325816320,"data_stored":325490695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215177,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807772,"kb_used_data":451272,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93564068,"statfs":{"total":96636764160,"available":95809605632,"internally_reserved":0,"allocated":462102528,"data_stored":461843597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460382208,"data_stored":460378729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325266915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385437696,"data_stored":385430145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:37.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:37.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11896", 'reported_seq': 11975, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.258176+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:36.258176+0000', 'last_peered': '2024-09-11T05:18:36.258176+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:36.258176+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33972224, 'num_objects': 8294, 'num_object_clones': 0, 'num_object_copies': 16588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11896, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4619}, {'shards': '0,2', 'objects': 1873}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:37.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11567", 'reported_seq': 11694, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.625783+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:36.625783+0000', 'last_peered': '2024-09-11T05:18:36.625783+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:36.625783+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33382424, 'num_objects': 8151, 'num_object_clones': 0, 'num_object_copies': 16302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8151, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9970, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1788}, {'shards': '2', 'objects': 4657}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11852", 'reported_seq': 16980, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.243019+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:36.243019+0000', 'last_peered': '2024-09-11T05:18:36.243019+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:36.243019+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33517568, 'num_objects': 8183, 'num_object_clones': 0, 'num_object_copies': 16366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1111, 'num_objects_unfound': 0, 'num_objects_dirty': 8183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8396, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2489, 'num_bytes_recovered': 10186752, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7070}, {'shards': '0,2', 'objects': 1111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11882", 'reported_seq': 12015, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.200362+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:36.200362+0000', 'last_peered': '2024-09-11T05:18:36.200362+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:36.200362+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34078720, 'num_objects': 8320, 'num_object_clones': 0, 'num_object_copies': 16640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8320, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9047, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4744}, {'shards': '0,2', 'objects': 1798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11733", 'reported_seq': 11865, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.644576+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:36.644576+0000', 'last_peered': '2024-09-11T05:18:36.644576+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:36.644576+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33742848, 'num_objects': 8238, 'num_object_clones': 0, 'num_object_copies': 16476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9801, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1781}, {'shards': '2', 'objects': 4713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11542", 'reported_seq': 11708, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.252981+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:36.252981+0000', 'last_peered': '2024-09-11T05:18:36.252981+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:36.252981+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33300480, 'num_objects': 8130, 'num_object_clones': 0, 'num_object_copies': 16260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8295, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4688}, {'shards': '0,2', 'objects': 1740}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11719", 'reported_seq': 11837, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.506514+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:36.506514+0000', 'last_peered': '2024-09-11T05:18:36.506514+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:36.506514+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34054144, 'num_objects': 8314, 'num_object_clones': 0, 'num_object_copies': 16628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9748, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1822}, {'shards': '2', 'objects': 4792}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11860", 'reported_seq': 11992, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.128942+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:36.128942+0000', 'last_peered': '2024-09-11T05:18:36.128942+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:36.128942+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33931264, 'num_objects': 8284, 'num_object_clones': 0, 'num_object_copies': 16568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8893, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4661}, {'shards': '0,2', 'objects': 1836}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11506", 'reported_seq': 11640, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.187181+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:36.187181+0000', 'last_peered': '2024-09-11T05:18:36.187181+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:36.187181+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33374208, 'num_objects': 8148, 'num_object_clones': 0, 'num_object_copies': 16296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8372, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4653}, {'shards': '0,2', 'objects': 1820}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11638", 'reported_seq': 11774, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.185979+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:36.185979+0000', 'last_peered': '2024-09-11T05:18:36.185979+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:36.185979+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33431552, 'num_objects': 8162, 'num_object_clones': 0, 'num_object_copies': 16324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8396, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4618}, {'shards': '0,2', 'objects': 1807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:37.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11578", 'reported_seq': 11722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:36.250354+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:36.250354+0000', 'last_peered': '2024-09-11T05:18:36.250354+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:36.250354+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32817769, 'num_objects': 8013, 'num_object_clones': 0, 'num_object_copies': 16026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8585, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4498}, {'shards': '0,2', 'objects': 1738}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:37.790 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:38.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:38.761 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/262402 objects degraded (13.503%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":396,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131201,"data_bytes":537383572,"bytes_used":2142777344,"bytes_avail":287767515136,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":262402,"degraded_ratio":0.13502564767036837,"misplaced_objects":1111,"misplaced_total":262402,"misplaced_ratio":0.004233961631390005,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92456,"recovering_keys_per_sec":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":562},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:38.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:38.835 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:39.713 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:39.805 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-11T05:18:39.520003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537383572,"num_objects":131201,"num_object_clones":0,"num_object_copies":262402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1111,"num_objects_unfound":0,"num_objects_dirty":131201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150498,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5971,"num_bytes_recovered":24436755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2083256,"kb_used_data":1145540,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281032264,"statfs":{"total":289910292480,"available":287777038336,"internally_reserved":0,"allocated":1173032960,"data_stored":1172189478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":960154625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,6,4,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18604032,"num_objects":-4542,"num_object_clones":0,"num_object_copies":-9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-168,"num_objects_unfound":0,"num_objects_dirty":-4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4542,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005845"},"pg_stats":[{"pgid":"2.f","version":"45'11896","reported_seq":11975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.258176+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:36.258176+0000","last_peered":"2024-09-11T05:18:36.258176+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:36.258176+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33972224,"num_objects":8294,"num_object_clones":0,"num_object_copies":16588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11896,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4619},{"shards":"0,2","objects":1873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11753","reported_seq":11856,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.567704+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:36.567704+0000","last_peered":"2024-09-11T05:18:36.567704+0000","last_clean":"2024-09-11T05:18:36.567704+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:36.567704+0000","last_undegraded":"2024-09-11T05:18:36.567704+0000","last_fullsized":"2024-09-11T05:18:36.567704+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33370112,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10151,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11567","reported_seq":11694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.625783+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:36.625783+0000","last_peered":"2024-09-11T05:18:36.625783+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:36.625783+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382424,"num_objects":8151,"num_object_clones":0,"num_object_copies":16302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9970,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1788},{"shards":"2","objects":4657}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11784","reported_seq":11855,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.601101+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:36.601101+0000","last_peered":"2024-09-11T05:18:36.601101+0000","last_clean":"2024-09-11T05:18:36.601101+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:36.601101+0000","last_undegraded":"2024-09-11T05:18:36.601101+0000","last_fullsized":"2024-09-11T05:18:36.601101+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33488896,"num_objects":8176,"num_object_clones":0,"num_object_copies":16352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11784,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11850","reported_seq":11980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.160346+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:36.160346+0000","last_peered":"2024-09-11T05:18:36.160346+0000","last_clean":"2024-09-11T05:18:36.160346+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:36.160346+0000","last_undegraded":"2024-09-11T05:18:36.160346+0000","last_fullsized":"2024-09-11T05:18:36.160346+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34074624,"num_objects":8319,"num_object_clones":0,"num_object_copies":16638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8684,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11852","reported_seq":16980,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.243019+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:36.243019+0000","last_peered":"2024-09-11T05:18:36.243019+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:36.243019+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33517568,"num_objects":8183,"num_object_clones":0,"num_object_copies":16366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1111,"num_objects_unfound":0,"num_objects_dirty":8183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8396,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2489,"num_bytes_recovered":10186752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7070},{"shards":"0,2","objects":1111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11882","reported_seq":12015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.200362+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:36.200362+0000","last_peered":"2024-09-11T05:18:36.200362+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:36.200362+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34078720,"num_objects":8320,"num_object_clones":0,"num_object_copies":16640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9047,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11694","reported_seq":11819,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.186527+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:36.186527+0000","last_peered":"2024-09-11T05:18:36.186527+0000","last_clean":"2024-09-11T05:18:36.186527+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:36.186527+0000","last_undegraded":"2024-09-11T05:18:36.186527+0000","last_fullsized":"2024-09-11T05:18:36.186527+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33546240,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8997,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11733","reported_seq":11865,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.644576+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:36.644576+0000","last_peered":"2024-09-11T05:18:36.644576+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:36.644576+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33742848,"num_objects":8238,"num_object_clones":0,"num_object_copies":16476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1781},{"shards":"2","objects":4713}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11542","reported_seq":11708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.252981+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:36.252981+0000","last_peered":"2024-09-11T05:18:36.252981+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:36.252981+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8295,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4688},{"shards":"0,2","objects":1740}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11719","reported_seq":11837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.506514+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:36.506514+0000","last_peered":"2024-09-11T05:18:36.506514+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:36.506514+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34054144,"num_objects":8314,"num_object_clones":0,"num_object_copies":16628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9748,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1822},{"shards":"2","objects":4792}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11578","reported_seq":18664,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:36.468989+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:36.468989+0000","last_peered":"2024-09-11T05:18:36.468989+0000","last_clean":"2024-09-11T05:18:36.468989+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:36.468989+0000","last_undegraded":"2024-09-11T05:18:36.468989+0000","last_fullsized":"2024-09-11T05:18:36.468989+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9481,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11860","reported_seq":11992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.128942+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:36.128942+0000","last_peered":"2024-09-11T05:18:36.128942+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:36.128942+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33931264,"num_objects":8284,"num_object_clones":0,"num_object_copies":16568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8893,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4661},{"shards":"0,2","objects":1836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11506","reported_seq":11640,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.187181+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:36.187181+0000","last_peered":"2024-09-11T05:18:36.187181+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:36.187181+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33374208,"num_objects":8148,"num_object_clones":0,"num_object_copies":16296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8372,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4653},{"shards":"0,2","objects":1820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11638","reported_seq":11774,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.185979+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:36.185979+0000","last_peered":"2024-09-11T05:18:36.185979+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:36.185979+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33431552,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8396,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4618},{"shards":"0,2","objects":1807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11578","reported_seq":11722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:36.250354+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:36.250354+0000","last_peered":"2024-09-11T05:18:36.250354+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:36.250354+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817769,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8585,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4498},{"shards":"0,2","objects":1738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537383553,"num_objects":131199,"num_object_clones":0,"num_object_copies":262398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1111,"num_objects_unfound":0,"num_objects_dirty":131199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150496,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5970,"num_bytes_recovered":24436736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1170214912,"data_stored":1170133709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673916,"kb_used_data":377068,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93697924,"statfs":{"total":96636764160,"available":95946674176,"internally_reserved":0,"allocated":386117632,"data_stored":385858706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601568,"kb_used_data":317200,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93770272,"statfs":{"total":96636764160,"available":96020758528,"internally_reserved":0,"allocated":324812800,"data_stored":324487175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215177,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807772,"kb_used_data":451272,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93564068,"statfs":{"total":96636764160,"available":95809605632,"internally_reserved":0,"allocated":462102528,"data_stored":461843597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":460382208,"data_stored":460378729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324395008,"data_stored":324324835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385437696,"data_stored":385430145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:42.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:43.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:43.762 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:43.835 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-11T05:18:43.523451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":528274068,"num_objects":128977,"num_object_clones":0,"num_object_copies":257954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":128977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152722,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6063,"num_bytes_recovered":24813587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2071372,"kb_used_data":1133648,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281044148,"statfs":{"total":289910292480,"available":287789207552,"internally_reserved":0,"allocated":1160855552,"data_stored":1160012070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":960154625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,6,8,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21970944,"num_objects":-5364,"num_object_clones":0,"num_object_copies":-10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-5364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5364,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008271"},"pg_stats":[{"pgid":"2.f","version":"45'12031","reported_seq":12110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.221803+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:41.221803+0000","last_peered":"2024-09-11T05:18:41.221803+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:41.221803+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12031,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1738}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11889","reported_seq":11992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.504733+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:41.504733+0000","last_peered":"2024-09-11T05:18:41.504733+0000","last_clean":"2024-09-11T05:18:41.504733+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:41.504733+0000","last_undegraded":"2024-09-11T05:18:41.504733+0000","last_fullsized":"2024-09-11T05:18:41.504733+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32813056,"num_objects":8011,"num_object_clones":0,"num_object_copies":16022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10287,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11701","reported_seq":11828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.595798+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:41.595798+0000","last_peered":"2024-09-11T05:18:41.595798+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:41.595798+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833560,"num_objects":8017,"num_object_clones":0,"num_object_copies":16034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10104,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1654},{"shards":"2","objects":4523}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11919","reported_seq":11991,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.637742+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:41.637742+0000","last_peered":"2024-09-11T05:18:41.637742+0000","last_clean":"2024-09-11T05:18:41.637742+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:41.637742+0000","last_undegraded":"2024-09-11T05:18:41.637742+0000","last_fullsized":"2024-09-11T05:18:41.637742+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11919,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11987","reported_seq":12117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.254895+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:41.254895+0000","last_peered":"2024-09-11T05:18:41.254895+0000","last_clean":"2024-09-11T05:18:41.254895+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:41.254895+0000","last_undegraded":"2024-09-11T05:18:41.254895+0000","last_fullsized":"2024-09-11T05:18:41.254895+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33513472,"num_objects":8182,"num_object_clones":0,"num_object_copies":16364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12000","reported_seq":17310,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.283720+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:41.283720+0000","last_peered":"2024-09-11T05:18:41.283720+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:41.283720+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32911360,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8544,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2581,"num_bytes_recovered":10563584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7014},{"shards":"0,2","objects":1019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12014","reported_seq":12147,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.229557+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:41.229557+0000","last_peered":"2024-09-11T05:18:41.229557+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:41.229557+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":1666}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11856","reported_seq":11981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.215486+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:41.215486+0000","last_peered":"2024-09-11T05:18:41.215486+0000","last_clean":"2024-09-11T05:18:41.215486+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:41.215486+0000","last_undegraded":"2024-09-11T05:18:41.215486+0000","last_fullsized":"2024-09-11T05:18:41.215486+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32882688,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9159,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11882","reported_seq":12013,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.682563+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:41.682563+0000","last_peered":"2024-09-11T05:18:41.682563+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:41.682563+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9950,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1632},{"shards":"2","objects":4564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11675","reported_seq":11840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.268531+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:41.268531+0000","last_peered":"2024-09-11T05:18:41.268531+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:41.268531+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32755712,"num_objects":7997,"num_object_clones":0,"num_object_copies":15994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8428,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4555},{"shards":"0,2","objects":1607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11870","reported_seq":11988,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.625354+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:41.625354+0000","last_peered":"2024-09-11T05:18:41.625354+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:41.625354+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33435648,"num_objects":8163,"num_object_clones":0,"num_object_copies":16326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9899,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1671},{"shards":"2","objects":4641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11722","reported_seq":18807,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.632782+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:41.632782+0000","last_peered":"2024-09-11T05:18:41.632782+0000","last_clean":"2024-09-11T05:18:41.632782+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:41.632782+0000","last_undegraded":"2024-09-11T05:18:41.632782+0000","last_fullsized":"2024-09-11T05:18:41.632782+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9625,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12009","reported_seq":12141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.129450+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:41.129450+0000","last_peered":"2024-09-11T05:18:41.129450+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:41.129450+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11628","reported_seq":11762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.259551+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:41.259551+0000","last_peered":"2024-09-11T05:18:41.259551+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:41.259551+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874496,"num_objects":8026,"num_object_clones":0,"num_object_copies":16052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8494,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":1698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11772","reported_seq":11908,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.208356+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:41.208356+0000","last_peered":"2024-09-11T05:18:41.208356+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:41.208356+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32882688,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8530,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11701","reported_seq":11845,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.287316+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:41.287316+0000","last_peered":"2024-09-11T05:18:41.287316+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:41.287316+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313961,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":528274049,"num_objects":128975,"num_object_clones":0,"num_object_copies":257950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":128975,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152720,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6062,"num_bytes_recovered":24813568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1158098944,"data_stored":1158017741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656844,"kb_used_data":372472,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93714996,"statfs":{"total":96636764160,"available":95964155904,"internally_reserved":0,"allocated":381411328,"data_stored":381152402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605052,"kb_used_data":316524,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93766788,"statfs":{"total":96636764160,"available":96017190912,"internally_reserved":0,"allocated":324120576,"data_stored":323794951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215178,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809476,"kb_used_data":444652,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93562364,"statfs":{"total":96636764160,"available":95807860736,"internally_reserved":0,"allocated":455323648,"data_stored":455064717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,8,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453652480,"data_stored":453649001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323698688,"data_stored":323628515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380747776,"data_stored":380740225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:43.837 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:44.755 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:44.757 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:44.799 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-11T05:18:43.523451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":528274068,"num_objects":128977,"num_object_clones":0,"num_object_copies":257954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":128977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152722,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6063,"num_bytes_recovered":24813587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2071372,"kb_used_data":1133648,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281044148,"statfs":{"total":289910292480,"available":287789207552,"internally_reserved":0,"allocated":1160855552,"data_stored":1160012070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":960154625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,6,8,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21970944,"num_objects":-5364,"num_object_clones":0,"num_object_copies":-10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-5364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5364,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008271"},"pg_stats":[{"pgid":"2.f","version":"45'12031","reported_seq":12110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.221803+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:41.221803+0000","last_peered":"2024-09-11T05:18:41.221803+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:41.221803+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12031,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1738}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11889","reported_seq":11992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.504733+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:41.504733+0000","last_peered":"2024-09-11T05:18:41.504733+0000","last_clean":"2024-09-11T05:18:41.504733+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:41.504733+0000","last_undegraded":"2024-09-11T05:18:41.504733+0000","last_fullsized":"2024-09-11T05:18:41.504733+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32813056,"num_objects":8011,"num_object_clones":0,"num_object_copies":16022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10287,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11701","reported_seq":11828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.595798+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:41.595798+0000","last_peered":"2024-09-11T05:18:41.595798+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:41.595798+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833560,"num_objects":8017,"num_object_clones":0,"num_object_copies":16034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10104,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1654},{"shards":"2","objects":4523}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11919","reported_seq":11991,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.637742+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:41.637742+0000","last_peered":"2024-09-11T05:18:41.637742+0000","last_clean":"2024-09-11T05:18:41.637742+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:41.637742+0000","last_undegraded":"2024-09-11T05:18:41.637742+0000","last_fullsized":"2024-09-11T05:18:41.637742+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11919,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11987","reported_seq":12117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.254895+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:41.254895+0000","last_peered":"2024-09-11T05:18:41.254895+0000","last_clean":"2024-09-11T05:18:41.254895+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:41.254895+0000","last_undegraded":"2024-09-11T05:18:41.254895+0000","last_fullsized":"2024-09-11T05:18:41.254895+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33513472,"num_objects":8182,"num_object_clones":0,"num_object_copies":16364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12000","reported_seq":17310,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.283720+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:41.283720+0000","last_peered":"2024-09-11T05:18:41.283720+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:41.283720+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32911360,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8544,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2581,"num_bytes_recovered":10563584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7014},{"shards":"0,2","objects":1019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12014","reported_seq":12147,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.229557+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:41.229557+0000","last_peered":"2024-09-11T05:18:41.229557+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:41.229557+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":1666}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11856","reported_seq":11981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.215486+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:41.215486+0000","last_peered":"2024-09-11T05:18:41.215486+0000","last_clean":"2024-09-11T05:18:41.215486+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:41.215486+0000","last_undegraded":"2024-09-11T05:18:41.215486+0000","last_fullsized":"2024-09-11T05:18:41.215486+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32882688,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9159,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11882","reported_seq":12013,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.682563+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:41.682563+0000","last_peered":"2024-09-11T05:18:41.682563+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:41.682563+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9950,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1632},{"shards":"2","objects":4564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11675","reported_seq":11840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.268531+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:41.268531+0000","last_peered":"2024-09-11T05:18:41.268531+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:41.268531+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32755712,"num_objects":7997,"num_object_clones":0,"num_object_copies":15994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8428,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4555},{"shards":"0,2","objects":1607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11870","reported_seq":11988,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.625354+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:41.625354+0000","last_peered":"2024-09-11T05:18:41.625354+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:41.625354+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33435648,"num_objects":8163,"num_object_clones":0,"num_object_copies":16326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9899,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1671},{"shards":"2","objects":4641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11722","reported_seq":18807,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.632782+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:41.632782+0000","last_peered":"2024-09-11T05:18:41.632782+0000","last_clean":"2024-09-11T05:18:41.632782+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:41.632782+0000","last_undegraded":"2024-09-11T05:18:41.632782+0000","last_fullsized":"2024-09-11T05:18:41.632782+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9625,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12009","reported_seq":12141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.129450+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:41.129450+0000","last_peered":"2024-09-11T05:18:41.129450+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:41.129450+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11628","reported_seq":11762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.259551+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:41.259551+0000","last_peered":"2024-09-11T05:18:41.259551+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:41.259551+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874496,"num_objects":8026,"num_object_clones":0,"num_object_copies":16052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8494,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":1698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11772","reported_seq":11908,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.208356+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:41.208356+0000","last_peered":"2024-09-11T05:18:41.208356+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:41.208356+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32882688,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8530,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11701","reported_seq":11845,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.287316+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:41.287316+0000","last_peered":"2024-09-11T05:18:41.287316+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:41.287316+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313961,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":528274049,"num_objects":128975,"num_object_clones":0,"num_object_copies":257950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":128975,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152720,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6062,"num_bytes_recovered":24813568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1158098944,"data_stored":1158017741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656844,"kb_used_data":372472,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93714996,"statfs":{"total":96636764160,"available":95964155904,"internally_reserved":0,"allocated":381411328,"data_stored":381152402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605052,"kb_used_data":316524,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93766788,"statfs":{"total":96636764160,"available":96017190912,"internally_reserved":0,"allocated":324120576,"data_stored":323794951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215178,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809476,"kb_used_data":444652,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93562364,"statfs":{"total":96636764160,"available":95807860736,"internally_reserved":0,"allocated":455323648,"data_stored":455064717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,8,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453652480,"data_stored":453649001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323698688,"data_stored":323628515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380747776,"data_stored":380740225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:44.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:44.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12031", 'reported_seq': 12110, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.221803+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:41.221803+0000', 'last_peered': '2024-09-11T05:18:41.221803+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:41.221803+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33419264, 'num_objects': 8159, 'num_object_clones': 0, 'num_object_copies': 16318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12031, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4484}, {'shards': '0,2', 'objects': 1738}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11701", 'reported_seq': 11828, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.595798+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:41.595798+0000', 'last_peered': '2024-09-11T05:18:41.595798+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:41.595798+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32833560, 'num_objects': 8017, 'num_object_clones': 0, 'num_object_copies': 16034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8017, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10104, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1654}, {'shards': '2', 'objects': 4523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12000", 'reported_seq': 17310, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.283720+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:41.283720+0000', 'last_peered': '2024-09-11T05:18:41.283720+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:41.283720+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32911360, 'num_objects': 8035, 'num_object_clones': 0, 'num_object_copies': 16070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1019, 'num_objects_unfound': 0, 'num_objects_dirty': 8035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8544, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2581, 'num_bytes_recovered': 10563584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7014}, {'shards': '0,2', 'objects': 1019}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12014", 'reported_seq': 12147, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.229557+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:41.229557+0000', 'last_peered': '2024-09-11T05:18:41.229557+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:41.229557+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33538048, 'num_objects': 8188, 'num_object_clones': 0, 'num_object_copies': 16376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8188, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9179, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4612}, {'shards': '0,2', 'objects': 1666}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11882", 'reported_seq': 12013, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.682563+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:41.682563+0000', 'last_peered': '2024-09-11T05:18:41.682563+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:41.682563+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33132544, 'num_objects': 8089, 'num_object_clones': 0, 'num_object_copies': 16178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9950, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1632}, {'shards': '2', 'objects': 4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11675", 'reported_seq': 11840, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.268531+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:41.268531+0000', 'last_peered': '2024-09-11T05:18:41.268531+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:41.268531+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32755712, 'num_objects': 7997, 'num_object_clones': 0, 'num_object_copies': 15994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7997, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8428, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4555}, {'shards': '0,2', 'objects': 1607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11870", 'reported_seq': 11988, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.625354+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:41.625354+0000', 'last_peered': '2024-09-11T05:18:41.625354+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:41.625354+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33435648, 'num_objects': 8163, 'num_object_clones': 0, 'num_object_copies': 16326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9899, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1671}, {'shards': '2', 'objects': 4641}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12009", 'reported_seq': 12141, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.129450+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:41.129450+0000', 'last_peered': '2024-09-11T05:18:41.129450+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:41.129450+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33320960, 'num_objects': 8135, 'num_object_clones': 0, 'num_object_copies': 16270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9042, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4512}, {'shards': '0,2', 'objects': 1687}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11628", 'reported_seq': 11762, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.259551+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:41.259551+0000', 'last_peered': '2024-09-11T05:18:41.259551+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:41.259551+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32874496, 'num_objects': 8026, 'num_object_clones': 0, 'num_object_copies': 16052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8494, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4531}, {'shards': '0,2', 'objects': 1698}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11772", 'reported_seq': 11908, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.208356+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:41.208356+0000', 'last_peered': '2024-09-11T05:18:41.208356+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:41.208356+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32882688, 'num_objects': 8028, 'num_object_clones': 0, 'num_object_copies': 16056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8028, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8530, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4484}, {'shards': '0,2', 'objects': 1673}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:44.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11701", 'reported_seq': 11845, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:41.287316+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:41.287316+0000', 'last_peered': '2024-09-11T05:18:41.287316+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:41.287316+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32313961, 'num_objects': 7890, 'num_object_clones': 0, 'num_object_copies': 15780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8708, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4375}, {'shards': '0,2', 'objects': 1615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:44.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:45.863 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:45.863 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/257954 objects degraded (13.735%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":403,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128977,"data_bytes":528274068,"bytes_used":2121084928,"bytes_avail":287789207552,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":257954,"degraded_ratio":0.13735394682772897,"misplaced_objects":1019,"misplaced_total":257954,"misplaced_ratio":0.0039503167231366831,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59024,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":374},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:45.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:45.950 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:46.827 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:46.890 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-11T05:18:45.523916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":528274068,"num_objects":128977,"num_object_clones":0,"num_object_copies":257954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":128977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152722,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6063,"num_bytes_recovered":24813587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2071372,"kb_used_data":1133648,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281044148,"statfs":{"total":289910292480,"available":287789207552,"internally_reserved":0,"allocated":1160855552,"data_stored":1160012070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":960154625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,6,8,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18415616,"num_objects":-4496,"num_object_clones":0,"num_object_copies":-8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005392"},"pg_stats":[{"pgid":"2.f","version":"45'12031","reported_seq":12110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.221803+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:41.221803+0000","last_peered":"2024-09-11T05:18:41.221803+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:41.221803+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12031,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1738}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11889","reported_seq":11992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.504733+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:41.504733+0000","last_peered":"2024-09-11T05:18:41.504733+0000","last_clean":"2024-09-11T05:18:41.504733+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:41.504733+0000","last_undegraded":"2024-09-11T05:18:41.504733+0000","last_fullsized":"2024-09-11T05:18:41.504733+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32813056,"num_objects":8011,"num_object_clones":0,"num_object_copies":16022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10287,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11701","reported_seq":11828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.595798+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:41.595798+0000","last_peered":"2024-09-11T05:18:41.595798+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:41.595798+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833560,"num_objects":8017,"num_object_clones":0,"num_object_copies":16034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10104,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1654},{"shards":"2","objects":4523}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11919","reported_seq":11991,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.637742+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:41.637742+0000","last_peered":"2024-09-11T05:18:41.637742+0000","last_clean":"2024-09-11T05:18:41.637742+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:41.637742+0000","last_undegraded":"2024-09-11T05:18:41.637742+0000","last_fullsized":"2024-09-11T05:18:41.637742+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32935936,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11919,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11987","reported_seq":12117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.254895+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:41.254895+0000","last_peered":"2024-09-11T05:18:41.254895+0000","last_clean":"2024-09-11T05:18:41.254895+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:41.254895+0000","last_undegraded":"2024-09-11T05:18:41.254895+0000","last_fullsized":"2024-09-11T05:18:41.254895+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33513472,"num_objects":8182,"num_object_clones":0,"num_object_copies":16364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12000","reported_seq":17310,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.283720+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:41.283720+0000","last_peered":"2024-09-11T05:18:41.283720+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:41.283720+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32911360,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8544,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2581,"num_bytes_recovered":10563584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7014},{"shards":"0,2","objects":1019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12014","reported_seq":12147,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.229557+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:41.229557+0000","last_peered":"2024-09-11T05:18:41.229557+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:41.229557+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":1666}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11856","reported_seq":11981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.215486+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:41.215486+0000","last_peered":"2024-09-11T05:18:41.215486+0000","last_clean":"2024-09-11T05:18:41.215486+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:41.215486+0000","last_undegraded":"2024-09-11T05:18:41.215486+0000","last_fullsized":"2024-09-11T05:18:41.215486+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32882688,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9159,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11882","reported_seq":12013,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.682563+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:41.682563+0000","last_peered":"2024-09-11T05:18:41.682563+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:41.682563+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9950,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1632},{"shards":"2","objects":4564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11675","reported_seq":11840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.268531+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:41.268531+0000","last_peered":"2024-09-11T05:18:41.268531+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:41.268531+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32755712,"num_objects":7997,"num_object_clones":0,"num_object_copies":15994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8428,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4555},{"shards":"0,2","objects":1607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11870","reported_seq":11988,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.625354+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:41.625354+0000","last_peered":"2024-09-11T05:18:41.625354+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:41.625354+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33435648,"num_objects":8163,"num_object_clones":0,"num_object_copies":16326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9899,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1671},{"shards":"2","objects":4641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11722","reported_seq":18807,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:41.632782+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:41.632782+0000","last_peered":"2024-09-11T05:18:41.632782+0000","last_clean":"2024-09-11T05:18:41.632782+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:41.632782+0000","last_undegraded":"2024-09-11T05:18:41.632782+0000","last_fullsized":"2024-09-11T05:18:41.632782+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9625,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12009","reported_seq":12141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.129450+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:41.129450+0000","last_peered":"2024-09-11T05:18:41.129450+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:41.129450+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9042,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11628","reported_seq":11762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.259551+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:41.259551+0000","last_peered":"2024-09-11T05:18:41.259551+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:41.259551+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874496,"num_objects":8026,"num_object_clones":0,"num_object_copies":16052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8494,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":1698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11772","reported_seq":11908,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.208356+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:41.208356+0000","last_peered":"2024-09-11T05:18:41.208356+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:41.208356+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32882688,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8530,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4484},{"shards":"0,2","objects":1673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11701","reported_seq":11845,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:41.287316+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:41.287316+0000","last_peered":"2024-09-11T05:18:41.287316+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:41.287316+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313961,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8708,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":528274049,"num_objects":128975,"num_object_clones":0,"num_object_copies":257950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":1019,"num_objects_unfound":0,"num_objects_dirty":128975,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152720,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6062,"num_bytes_recovered":24813568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1158098944,"data_stored":1158017741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656844,"kb_used_data":372472,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93714996,"statfs":{"total":96636764160,"available":95964155904,"internally_reserved":0,"allocated":381411328,"data_stored":381152402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605052,"kb_used_data":316524,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93766788,"statfs":{"total":96636764160,"available":96017190912,"internally_reserved":0,"allocated":324120576,"data_stored":323794951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215178,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809476,"kb_used_data":444652,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93562364,"statfs":{"total":96636764160,"available":95807860736,"internally_reserved":0,"allocated":455323648,"data_stored":455064717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,8,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453652480,"data_stored":453649001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323698688,"data_stored":323628515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380747776,"data_stored":380740225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:49.892 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:50.711 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:50.711 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:50.751 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-11T05:18:49.524916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519185044,"num_objects":126758,"num_object_clones":0,"num_object_copies":253516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":900,"num_objects_unfound":0,"num_objects_dirty":126758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154941,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6182,"num_bytes_recovered":25301011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2059260,"kb_used_data":1121548,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281056260,"statfs":{"total":289910292480,"available":287801610240,"internally_reserved":0,"allocated":1148465152,"data_stored":1147621670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":960154625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,2,6,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18198528,"num_objects":-4443,"num_object_clones":0,"num_object_copies":-8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-211,"num_objects_unfound":0,"num_objects_dirty":-4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":864256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005404"},"pg_stats":[{"pgid":"2.f","version":"45'12193","reported_seq":12272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.294079+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:46.294079+0000","last_peered":"2024-09-11T05:18:46.294079+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:46.294079+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32755712,"num_objects":7997,"num_object_clones":0,"num_object_copies":15994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12193,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4322},{"shards":"0,2","objects":1576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12017","reported_seq":12120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.616549+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:46.616549+0000","last_peered":"2024-09-11T05:18:46.616549+0000","last_clean":"2024-09-11T05:18:46.616549+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:46.616549+0000","last_undegraded":"2024-09-11T05:18:46.616549+0000","last_fullsized":"2024-09-11T05:18:46.616549+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32288768,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10415,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11835","reported_seq":11962,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.695933+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:46.695933+0000","last_peered":"2024-09-11T05:18:46.695933+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:46.695933+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32284696,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10238,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1520},{"shards":"2","objects":4389}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12061","reported_seq":12133,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.696355+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:46.696355+0000","last_peered":"2024-09-11T05:18:46.696355+0000","last_clean":"2024-09-11T05:18:46.696355+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:46.696355+0000","last_undegraded":"2024-09-11T05:18:46.696355+0000","last_fullsized":"2024-09-11T05:18:46.696355+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32354304,"num_objects":7899,"num_object_clones":0,"num_object_copies":15798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12061,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12154","reported_seq":12282,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.304058+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:46.304058+0000","last_peered":"2024-09-11T05:18:46.304058+0000","last_clean":"2024-09-11T05:18:46.304058+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:46.304058+0000","last_undegraded":"2024-09-11T05:18:46.304058+0000","last_fullsized":"2024-09-11T05:18:46.304058+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32829440,"num_objects":8015,"num_object_clones":0,"num_object_copies":16030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8988,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12128","reported_seq":17679,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.297744+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:46.297744+0000","last_peered":"2024-09-11T05:18:46.297744+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:46.297744+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32387072,"num_objects":7907,"num_object_clones":0,"num_object_copies":15814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":900,"num_objects_unfound":0,"num_objects_dirty":7907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2700,"num_bytes_recovered":11051008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7005},{"shards":"0,2","objects":900}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12154","reported_seq":12287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.283782+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:46.283782+0000","last_peered":"2024-09-11T05:18:46.283782+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:46.283782+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32964608,"num_objects":8048,"num_object_clones":0,"num_object_copies":16096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9319,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4472},{"shards":"0,2","objects":1526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11994","reported_seq":12119,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.301747+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:46.301747+0000","last_peered":"2024-09-11T05:18:46.301747+0000","last_clean":"2024-09-11T05:18:46.301747+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:46.301747+0000","last_undegraded":"2024-09-11T05:18:46.301747+0000","last_fullsized":"2024-09-11T05:18:46.301747+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12029","reported_seq":12160,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.708686+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:46.708686+0000","last_peered":"2024-09-11T05:18:46.708686+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:46.708686+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32530432,"num_objects":7942,"num_object_clones":0,"num_object_copies":15884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10097,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1485},{"shards":"2","objects":4417}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11787","reported_seq":11953,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.214772+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:46.214772+0000","last_peered":"2024-09-11T05:18:46.214772+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:46.214772+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32296960,"num_objects":7885,"num_object_clones":0,"num_object_copies":15770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8540,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4443},{"shards":"0,2","objects":1495}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12007","reported_seq":12125,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.417629+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:46.417629+0000","last_peered":"2024-09-11T05:18:46.417629+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:46.417629+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874496,"num_objects":8026,"num_object_clones":0,"num_object_copies":16052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1534},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11877","reported_seq":18963,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.648779+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:46.648779+0000","last_peered":"2024-09-11T05:18:46.648779+0000","last_clean":"2024-09-11T05:18:46.648779+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:46.648779+0000","last_undegraded":"2024-09-11T05:18:46.648779+0000","last_fullsized":"2024-09-11T05:18:46.648779+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075776,"num_objects":7831,"num_object_clones":0,"num_object_copies":15662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9780,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12136","reported_seq":12267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.295472+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:46.295472+0000","last_peered":"2024-09-11T05:18:46.295472+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:46.295472+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32800768,"num_objects":8008,"num_object_clones":0,"num_object_copies":16016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4385},{"shards":"0,2","objects":1560}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11773","reported_seq":11904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.289677+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:46.289677+0000","last_peered":"2024-09-11T05:18:46.289677+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:46.289677+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32280576,"num_objects":7881,"num_object_clones":0,"num_object_copies":15762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8639,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4386},{"shards":"0,2","objects":1553}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11908","reported_seq":12044,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.264253+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:46.264253+0000","last_peered":"2024-09-11T05:18:46.264253+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:46.264253+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32325632,"num_objects":7892,"num_object_clones":0,"num_object_copies":15784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8666,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":1537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11822","reported_seq":11966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.252359+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:46.252359+0000","last_peered":"2024-09-11T05:18:46.252359+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:46.252359+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31818345,"num_objects":7769,"num_object_clones":0,"num_object_copies":15538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8829,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4254},{"shards":"0,2","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":519185025,"num_objects":126756,"num_object_clones":0,"num_object_copies":253512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":900,"num_objects_unfound":0,"num_objects_dirty":126756,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154939,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6181,"num_bytes_recovered":25300992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1145778176,"data_stored":1145696973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656356,"kb_used_data":367828,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93715484,"statfs":{"total":96636764160,"available":95964655616,"internally_reserved":0,"allocated":376655872,"data_stored":376396946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612588,"kb_used_data":315740,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93759252,"statfs":{"total":96636764160,"available":96009474048,"internally_reserved":0,"allocated":323317760,"data_stored":322992135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215179,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790316,"kb_used_data":437980,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93581524,"statfs":{"total":96636764160,"available":95827480576,"internally_reserved":0,"allocated":448491520,"data_stored":448232589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360771243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446873600,"data_stored":446870121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322850816,"data_stored":322780643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376053760,"data_stored":376046209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:50.753 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:51.785 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:51.785 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:51.835 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-11T05:18:51.525373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":513507988,"num_objects":125372,"num_object_clones":0,"num_object_copies":250744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":125372,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156327,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6292,"num_bytes_recovered":25751571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2059748,"kb_used_data":1113716,"kb_used_omap":12,"kb_used_meta":945971,"kb_avail":281055772,"statfs":{"total":289910292480,"available":287801110528,"internally_reserved":0,"allocated":1140445184,"data_stored":1139601702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":968674305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,2,9,5,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23875584,"num_objects":-5829,"num_object_clones":0,"num_object_copies":-11658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":-5829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5829,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005369"},"pg_stats":[{"pgid":"2.f","version":"45'12327","reported_seq":12406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.368311+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:51.368311+0000","last_peered":"2024-09-11T05:18:51.368311+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:51.368311+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12327,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4188},{"shards":"0,2","objects":1442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12017","reported_seq":12120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.616549+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:46.616549+0000","last_peered":"2024-09-11T05:18:46.616549+0000","last_clean":"2024-09-11T05:18:46.616549+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:46.616549+0000","last_undegraded":"2024-09-11T05:18:46.616549+0000","last_fullsized":"2024-09-11T05:18:46.616549+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32288768,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10415,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11835","reported_seq":11962,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.695933+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:46.695933+0000","last_peered":"2024-09-11T05:18:46.695933+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:46.695933+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32284696,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10238,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1520},{"shards":"2","objects":4389}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12061","reported_seq":12133,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.696355+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:46.696355+0000","last_peered":"2024-09-11T05:18:46.696355+0000","last_clean":"2024-09-11T05:18:46.696355+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:46.696355+0000","last_undegraded":"2024-09-11T05:18:46.696355+0000","last_fullsized":"2024-09-11T05:18:46.696355+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32354304,"num_objects":7899,"num_object_clones":0,"num_object_copies":15798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12061,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12294","reported_seq":12424,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.317003+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:51.317003+0000","last_peered":"2024-09-11T05:18:51.317003+0000","last_clean":"2024-09-11T05:18:51.317003+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:51.317003+0000","last_undegraded":"2024-09-11T05:18:51.317003+0000","last_fullsized":"2024-09-11T05:18:51.317003+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12275","reported_seq":18044,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.381777+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:51.381777+0000","last_peered":"2024-09-11T05:18:51.381777+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:51.381777+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":7760,"num_object_clones":0,"num_object_copies":15520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":7760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8819,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2810,"num_bytes_recovered":11501568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6968},{"shards":"0,2","objects":790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12312","reported_seq":12443,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.333585+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:51.333585+0000","last_peered":"2024-09-11T05:18:51.333585+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:51.333585+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4314},{"shards":"0,2","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12120","reported_seq":12245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.275938+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:51.275938+0000","last_peered":"2024-09-11T05:18:51.275938+0000","last_clean":"2024-09-11T05:18:51.275938+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:51.275938+0000","last_undegraded":"2024-09-11T05:18:51.275938+0000","last_fullsized":"2024-09-11T05:18:51.275938+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9423,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12029","reported_seq":12160,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.708686+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:46.708686+0000","last_peered":"2024-09-11T05:18:46.708686+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:46.708686+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32530432,"num_objects":7942,"num_object_clones":0,"num_object_copies":15884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10097,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1485},{"shards":"2","objects":4417}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11915","reported_seq":12081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.373604+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:51.373604+0000","last_peered":"2024-09-11T05:18:51.373604+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:51.373604+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8668,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":1367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12007","reported_seq":12125,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:46.417629+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:46.417629+0000","last_peered":"2024-09-11T05:18:46.417629+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:46.417629+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874496,"num_objects":8026,"num_object_clones":0,"num_object_copies":16052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1534},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11877","reported_seq":18963,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:46.648779+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:46.648779+0000","last_peered":"2024-09-11T05:18:46.648779+0000","last_clean":"2024-09-11T05:18:46.648779+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:46.648779+0000","last_undegraded":"2024-09-11T05:18:46.648779+0000","last_fullsized":"2024-09-11T05:18:46.648779+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075776,"num_objects":7831,"num_object_clones":0,"num_object_copies":15662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9780,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12275","reported_seq":12407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.276820+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:51.276820+0000","last_peered":"2024-09-11T05:18:51.276820+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:51.276820+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231424,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4246},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11922","reported_seq":12054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.366242+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:51.366242+0000","last_peered":"2024-09-11T05:18:51.366242+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:51.366242+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237},{"shards":"0,2","objects":1404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12052","reported_seq":12188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.278698+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:51.278698+0000","last_peered":"2024-09-11T05:18:51.278698+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:51.278698+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8810,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4204},{"shards":"0,2","objects":1393}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11943","reported_seq":12087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.368117+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:51.368117+0000","last_peered":"2024-09-11T05:18:51.368117+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:51.368117+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31322729,"num_objects":7648,"num_object_clones":0,"num_object_copies":15296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8950,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":513507969,"num_objects":125370,"num_object_clones":0,"num_object_copies":250740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":125370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156325,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6291,"num_bytes_recovered":25751552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1138946048,"data_stored":1138864845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160661,"ondisk_log_size":160661,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656356,"kb_used_data":367828,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93715484,"statfs":{"total":96636764160,"available":95964655616,"internally_reserved":0,"allocated":376655872,"data_stored":376396946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612588,"kb_used_data":315740,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93759252,"statfs":{"total":96636764160,"available":96009474048,"internally_reserved":0,"allocated":323317760,"data_stored":322992135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215180,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790804,"kb_used_data":430148,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93581036,"statfs":{"total":96636764160,"available":95826980864,"internally_reserved":0,"allocated":440471552,"data_stored":440212621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,4,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440041472,"data_stored":440037993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322850816,"data_stored":322780643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376053760,"data_stored":376046209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:51.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:51.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12327", 'reported_seq': 12406, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.368311+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:51.368311+0000', 'last_peered': '2024-09-11T05:18:51.368311+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:51.368311+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32206848, 'num_objects': 7863, 'num_object_clones': 0, 'num_object_copies': 15726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12327, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4188}, {'shards': '0,2', 'objects': 1442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:51.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11835", 'reported_seq': 11962, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:46.695933+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:46.695933+0000', 'last_peered': '2024-09-11T05:18:46.695933+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:46.695933+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32284696, 'num_objects': 7883, 'num_object_clones': 0, 'num_object_copies': 15766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7883, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10238, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1520}, {'shards': '2', 'objects': 4389}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:51.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:51.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12275", 'reported_seq': 18044, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.381777+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:51.381777+0000', 'last_peered': '2024-09-11T05:18:51.381777+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:51.381777+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31784960, 'num_objects': 7760, 'num_object_clones': 0, 'num_object_copies': 15520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 790, 'num_objects_unfound': 0, 'num_objects_dirty': 7760, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8819, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2810, 'num_bytes_recovered': 11501568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6968}, {'shards': '0,2', 'objects': 790}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12312", 'reported_seq': 12443, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.333585+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:51.333585+0000', 'last_peered': '2024-09-11T05:18:51.333585+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:51.333585+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32317440, 'num_objects': 7890, 'num_object_clones': 0, 'num_object_copies': 15780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9477, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4314}, {'shards': '0,2', 'objects': 1368}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12029", 'reported_seq': 12160, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:46.708686+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:46.708686+0000', 'last_peered': '2024-09-11T05:18:46.708686+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:46.708686+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32530432, 'num_objects': 7942, 'num_object_clones': 0, 'num_object_copies': 15884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7942, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10097, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1485}, {'shards': '2', 'objects': 4417}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11915", 'reported_seq': 12081, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.373604+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:51.373604+0000', 'last_peered': '2024-09-11T05:18:51.373604+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:51.373604+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31772672, 'num_objects': 7757, 'num_object_clones': 0, 'num_object_copies': 15514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7757, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8668, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4315}, {'shards': '0,2', 'objects': 1367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12007", 'reported_seq': 12125, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:46.417629+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:46.417629+0000', 'last_peered': '2024-09-11T05:18:46.417629+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:46.417629+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32874496, 'num_objects': 8026, 'num_object_clones': 0, 'num_object_copies': 16052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10036, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1534}, {'shards': '2', 'objects': 4504}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12275", 'reported_seq': 12407, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.276820+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:51.276820+0000', 'last_peered': '2024-09-11T05:18:51.276820+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:51.276820+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32231424, 'num_objects': 7869, 'num_object_clones': 0, 'num_object_copies': 15738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9308, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4246}, {'shards': '0,2', 'objects': 1421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:51.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11922", 'reported_seq': 12054, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.366242+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:51.366242+0000', 'last_peered': '2024-09-11T05:18:51.366242+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:51.366242+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31670272, 'num_objects': 7732, 'num_object_clones': 0, 'num_object_copies': 15464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8788, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4237}, {'shards': '0,2', 'objects': 1404}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:51.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12052", 'reported_seq': 12188, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.278698+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:51.278698+0000', 'last_peered': '2024-09-11T05:18:51.278698+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:51.278698+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31735808, 'num_objects': 7748, 'num_object_clones': 0, 'num_object_copies': 15496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8810, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4204}, {'shards': '0,2', 'objects': 1393}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:51.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11943", 'reported_seq': 12087, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:51.368117+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:51.368117+0000', 'last_peered': '2024-09-11T05:18:51.368117+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:51.368117+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31322729, 'num_objects': 7648, 'num_object_clones': 0, 'num_object_copies': 15296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7648, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8950, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4133}, {'shards': '0,2', 'objects': 1373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:51.839 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:52.811 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:52.811 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/250744 objects degraded (14.130%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125372,"data_bytes":513507988,"bytes_used":2109181952,"bytes_avail":287801110528,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":250744,"degraded_ratio":0.14130348084101713,"misplaced_objects":790,"misplaced_total":250744,"misplaced_ratio":0.003150623743738634,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":109518,"recovering_keys_per_sec":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":485},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:52.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:52.868 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:53.594 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:53.595 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:53.671 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-11T05:18:53.525861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":509919892,"num_objects":124496,"num_object_clones":0,"num_object_copies":248992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":124496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157203,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6292,"num_bytes_recovered":25751571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046240,"kb_used_data":1108528,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281069280,"statfs":{"total":289910292480,"available":287814942720,"internally_reserved":0,"allocated":1135132672,"data_stored":1134289190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":960154625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,11,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21831680,"num_objects":-5330,"num_object_clones":0,"num_object_copies":-10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":-5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5330,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002899"},"pg_stats":[{"pgid":"2.f","version":"45'12327","reported_seq":12406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.368311+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:51.368311+0000","last_peered":"2024-09-11T05:18:51.368311+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:51.368311+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12327,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4188},{"shards":"0,2","objects":1442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12154","reported_seq":12257,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.652338+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:51.652338+0000","last_peered":"2024-09-11T05:18:51.652338+0000","last_clean":"2024-09-11T05:18:51.652338+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:51.652338+0000","last_undegraded":"2024-09-11T05:18:51.652338+0000","last_fullsized":"2024-09-11T05:18:51.652338+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31727616,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11974","reported_seq":12100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.770384+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:51.770384+0000","last_peered":"2024-09-11T05:18:51.770384+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:51.770384+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31715352,"num_objects":7744,"num_object_clones":0,"num_object_copies":15488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10377,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":4250}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12203","reported_seq":12275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.652329+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:51.652329+0000","last_peered":"2024-09-11T05:18:51.652329+0000","last_clean":"2024-09-11T05:18:51.652329+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:51.652329+0000","last_undegraded":"2024-09-11T05:18:51.652329+0000","last_fullsized":"2024-09-11T05:18:51.652329+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12203,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12294","reported_seq":12424,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.317003+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:51.317003+0000","last_peered":"2024-09-11T05:18:51.317003+0000","last_clean":"2024-09-11T05:18:51.317003+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:51.317003+0000","last_undegraded":"2024-09-11T05:18:51.317003+0000","last_fullsized":"2024-09-11T05:18:51.317003+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12275","reported_seq":18044,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.381777+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:51.381777+0000","last_peered":"2024-09-11T05:18:51.381777+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:51.381777+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":7760,"num_object_clones":0,"num_object_copies":15520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":7760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8819,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2810,"num_bytes_recovered":11501568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6968},{"shards":"0,2","objects":790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12312","reported_seq":12443,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.333585+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:51.333585+0000","last_peered":"2024-09-11T05:18:51.333585+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:51.333585+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4314},{"shards":"0,2","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12120","reported_seq":12245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.275938+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:51.275938+0000","last_peered":"2024-09-11T05:18:51.275938+0000","last_clean":"2024-09-11T05:18:51.275938+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:51.275938+0000","last_undegraded":"2024-09-11T05:18:51.275938+0000","last_fullsized":"2024-09-11T05:18:51.275938+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9423,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12179","reported_seq":12311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.734078+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:51.734078+0000","last_peered":"2024-09-11T05:18:51.734078+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:51.734078+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10247,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1335},{"shards":"2","objects":4267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11915","reported_seq":12081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.373604+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:51.373604+0000","last_peered":"2024-09-11T05:18:51.373604+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:51.373604+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8668,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":1367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12161","reported_seq":12279,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.620955+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:51.620955+0000","last_peered":"2024-09-11T05:18:51.620955+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:51.620955+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32243712,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10190,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1380},{"shards":"2","objects":4350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12031","reported_seq":19117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.658405+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:51.658405+0000","last_peered":"2024-09-11T05:18:51.658405+0000","last_clean":"2024-09-11T05:18:51.658405+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:51.658405+0000","last_undegraded":"2024-09-11T05:18:51.658405+0000","last_fullsized":"2024-09-11T05:18:51.658405+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9934,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12275","reported_seq":12407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.276820+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:51.276820+0000","last_peered":"2024-09-11T05:18:51.276820+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:51.276820+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231424,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4246},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11922","reported_seq":12054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.366242+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:51.366242+0000","last_peered":"2024-09-11T05:18:51.366242+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:51.366242+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237},{"shards":"0,2","objects":1404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12052","reported_seq":12188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.278698+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:51.278698+0000","last_peered":"2024-09-11T05:18:51.278698+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:51.278698+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8810,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4204},{"shards":"0,2","objects":1393}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11943","reported_seq":12087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.368117+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:51.368117+0000","last_peered":"2024-09-11T05:18:51.368117+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:51.368117+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31322729,"num_objects":7648,"num_object_clones":0,"num_object_copies":15296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8950,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":509919873,"num_objects":124494,"num_object_clones":0,"num_object_copies":248988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":124494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157201,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6291,"num_bytes_recovered":25751552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1133600768,"data_stored":1133519565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656048,"kb_used_data":363360,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93715792,"statfs":{"total":96636764160,"available":95964971008,"internally_reserved":0,"allocated":372080640,"data_stored":371821714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599388,"kb_used_data":315020,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93772452,"statfs":{"total":96636764160,"available":96022990848,"internally_reserved":0,"allocated":322580480,"data_stored":322254855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215180,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790804,"kb_used_data":430148,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93581036,"statfs":{"total":96636764160,"available":95826980864,"internally_reserved":0,"allocated":440471552,"data_stored":440212621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,4,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440041472,"data_stored":440037993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322199552,"data_stored":322129379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371359744,"data_stored":371352193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:56.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:57.290 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:57.333 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-11T05:18:55.526374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":509919892,"num_objects":124496,"num_object_clones":0,"num_object_copies":248992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":124496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157203,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6292,"num_bytes_recovered":25751571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046240,"kb_used_data":1108528,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281069280,"statfs":{"total":289910292480,"available":287814942720,"internally_reserved":0,"allocated":1135132672,"data_stored":1134289190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":960154625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,11,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18354176,"num_objects":-4481,"num_object_clones":0,"num_object_copies":-8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":-4481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4481,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002923"},"pg_stats":[{"pgid":"2.f","version":"45'12327","reported_seq":12406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.368311+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:51.368311+0000","last_peered":"2024-09-11T05:18:51.368311+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:51.368311+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12327,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4188},{"shards":"0,2","objects":1442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12154","reported_seq":12257,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.652338+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:51.652338+0000","last_peered":"2024-09-11T05:18:51.652338+0000","last_clean":"2024-09-11T05:18:51.652338+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:51.652338+0000","last_undegraded":"2024-09-11T05:18:51.652338+0000","last_fullsized":"2024-09-11T05:18:51.652338+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31727616,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11974","reported_seq":12100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.770384+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:51.770384+0000","last_peered":"2024-09-11T05:18:51.770384+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:51.770384+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31715352,"num_objects":7744,"num_object_clones":0,"num_object_copies":15488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10377,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":4250}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12203","reported_seq":12275,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.652329+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:51.652329+0000","last_peered":"2024-09-11T05:18:51.652329+0000","last_clean":"2024-09-11T05:18:51.652329+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:51.652329+0000","last_undegraded":"2024-09-11T05:18:51.652329+0000","last_fullsized":"2024-09-11T05:18:51.652329+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12203,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12294","reported_seq":12424,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.317003+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:51.317003+0000","last_peered":"2024-09-11T05:18:51.317003+0000","last_clean":"2024-09-11T05:18:51.317003+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:51.317003+0000","last_undegraded":"2024-09-11T05:18:51.317003+0000","last_fullsized":"2024-09-11T05:18:51.317003+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12275","reported_seq":18044,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.381777+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:51.381777+0000","last_peered":"2024-09-11T05:18:51.381777+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:51.381777+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":7760,"num_object_clones":0,"num_object_copies":15520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":7760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8819,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2810,"num_bytes_recovered":11501568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6968},{"shards":"0,2","objects":790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12312","reported_seq":12443,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.333585+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:51.333585+0000","last_peered":"2024-09-11T05:18:51.333585+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:51.333585+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4314},{"shards":"0,2","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12120","reported_seq":12245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.275938+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:51.275938+0000","last_peered":"2024-09-11T05:18:51.275938+0000","last_clean":"2024-09-11T05:18:51.275938+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:51.275938+0000","last_undegraded":"2024-09-11T05:18:51.275938+0000","last_fullsized":"2024-09-11T05:18:51.275938+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9423,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12179","reported_seq":12311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.734078+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:51.734078+0000","last_peered":"2024-09-11T05:18:51.734078+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:51.734078+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10247,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1335},{"shards":"2","objects":4267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11915","reported_seq":12081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.373604+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:51.373604+0000","last_peered":"2024-09-11T05:18:51.373604+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:51.373604+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8668,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":1367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12161","reported_seq":12279,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.620955+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:51.620955+0000","last_peered":"2024-09-11T05:18:51.620955+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:51.620955+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32243712,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10190,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1380},{"shards":"2","objects":4350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12031","reported_seq":19117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:51.658405+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:51.658405+0000","last_peered":"2024-09-11T05:18:51.658405+0000","last_clean":"2024-09-11T05:18:51.658405+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:51.658405+0000","last_undegraded":"2024-09-11T05:18:51.658405+0000","last_fullsized":"2024-09-11T05:18:51.658405+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9934,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12275","reported_seq":12407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.276820+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:51.276820+0000","last_peered":"2024-09-11T05:18:51.276820+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:51.276820+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231424,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4246},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11922","reported_seq":12054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.366242+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:51.366242+0000","last_peered":"2024-09-11T05:18:51.366242+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:51.366242+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237},{"shards":"0,2","objects":1404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12052","reported_seq":12188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.278698+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:51.278698+0000","last_peered":"2024-09-11T05:18:51.278698+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:51.278698+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8810,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4204},{"shards":"0,2","objects":1393}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11943","reported_seq":12087,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:51.368117+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:51.368117+0000","last_peered":"2024-09-11T05:18:51.368117+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:51.368117+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31322729,"num_objects":7648,"num_object_clones":0,"num_object_copies":15296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8950,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":509919873,"num_objects":124494,"num_object_clones":0,"num_object_copies":248988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":790,"num_objects_unfound":0,"num_objects_dirty":124494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157201,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6291,"num_bytes_recovered":25751552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1133600768,"data_stored":1133519565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656048,"kb_used_data":363360,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93715792,"statfs":{"total":96636764160,"available":95964971008,"internally_reserved":0,"allocated":372080640,"data_stored":371821714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599388,"kb_used_data":315020,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93772452,"statfs":{"total":96636764160,"available":96022990848,"internally_reserved":0,"allocated":322580480,"data_stored":322254855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215180,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790804,"kb_used_data":430148,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93581036,"statfs":{"total":96636764160,"available":95826980864,"internally_reserved":0,"allocated":440471552,"data_stored":440212621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,4,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440041472,"data_stored":440037993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322199552,"data_stored":322129379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371359744,"data_stored":371352193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:57.335 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:18:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:58.281 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:18:58.314 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-11T05:18:57.526818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":500789908,"num_objects":122267,"num_object_clones":0,"num_object_copies":244534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":682,"num_objects_unfound":0,"num_objects_dirty":122267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159432,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6400,"num_bytes_recovered":26193939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2043516,"kb_used_data":1097484,"kb_used_omap":12,"kb_used_meta":945971,"kb_avail":281072004,"statfs":{"total":289910292480,"available":287817732096,"internally_reserved":0,"allocated":1123823616,"data_stored":1122980134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":968674305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,9,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27484160,"num_objects":-6710,"num_object_clones":0,"num_object_copies":-13420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":-6710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6710,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002901"},"pg_stats":[{"pgid":"2.f","version":"45'12454","reported_seq":12533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.276041+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:56.276041+0000","last_peered":"2024-09-11T05:18:56.276041+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:56.276041+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12454,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":1315}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12284","reported_seq":12387,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.625366+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:56.625366+0000","last_peered":"2024-09-11T05:18:56.625366+0000","last_clean":"2024-09-11T05:18:56.625366+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:56.625366+0000","last_undegraded":"2024-09-11T05:18:56.625366+0000","last_fullsized":"2024-09-11T05:18:56.625366+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10682,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12109","reported_seq":12236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.724924+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:56.724924+0000","last_peered":"2024-09-11T05:18:56.724924+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:56.724924+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162392,"num_objects":7609,"num_object_clones":0,"num_object_copies":15218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10512,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1246},{"shards":"2","objects":4115}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12348","reported_seq":12420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.678398+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:56.678398+0000","last_peered":"2024-09-11T05:18:56.678398+0000","last_clean":"2024-09-11T05:18:56.678398+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:56.678398+0000","last_undegraded":"2024-09-11T05:18:56.678398+0000","last_fullsized":"2024-09-11T05:18:56.678398+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31178752,"num_objects":7612,"num_object_clones":0,"num_object_copies":15224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12425","reported_seq":12555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.362129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:56.362129+0000","last_peered":"2024-09-11T05:18:56.362129+0000","last_clean":"2024-09-11T05:18:56.362129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:56.362129+0000","last_undegraded":"2024-09-11T05:18:56.362129+0000","last_fullsized":"2024-09-11T05:18:56.362129+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":7744,"num_object_clones":0,"num_object_copies":15488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9259,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12423","reported_seq":18406,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.461864+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:56.461864+0000","last_peered":"2024-09-11T05:18:56.461864+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:56.461864+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31178752,"num_objects":7612,"num_object_clones":0,"num_object_copies":15224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":682,"num_objects_unfound":0,"num_objects_dirty":7612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8967,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2918,"num_bytes_recovered":11943936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6928},{"shards":"0,2","objects":682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12455","reported_seq":12588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.406059+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:56.406059+0000","last_peered":"2024-09-11T05:18:56.406059+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:56.406059+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31731712,"num_objects":7747,"num_object_clones":0,"num_object_copies":15494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9620,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4171},{"shards":"0,2","objects":1225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12246","reported_seq":12371,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.259398+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:56.259398+0000","last_peered":"2024-09-11T05:18:56.259398+0000","last_clean":"2024-09-11T05:18:56.259398+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:56.259398+0000","last_undegraded":"2024-09-11T05:18:56.259398+0000","last_fullsized":"2024-09-11T05:18:56.259398+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12319","reported_seq":12450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.749278+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:56.749278+0000","last_peered":"2024-09-11T05:18:56.749278+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:56.749278+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31342592,"num_objects":7652,"num_object_clones":0,"num_object_copies":15304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10387,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1195},{"shards":"2","objects":4127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12064","reported_seq":12228,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.352599+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:56.352599+0000","last_peered":"2024-09-11T05:18:56.352599+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:56.352599+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8817,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4166},{"shards":"0,2","objects":1218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12286","reported_seq":12403,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.735782+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:56.735782+0000","last_peered":"2024-09-11T05:18:56.735782+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:56.735782+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31731712,"num_objects":7747,"num_object_clones":0,"num_object_copies":15494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10315,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1255},{"shards":"2","objects":4225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12171","reported_seq":19257,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.695089+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:56.695089+0000","last_peered":"2024-09-11T05:18:56.695089+0000","last_clean":"2024-09-11T05:18:56.695089+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:56.695089+0000","last_undegraded":"2024-09-11T05:18:56.695089+0000","last_fullsized":"2024-09-11T05:18:56.695089+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30871552,"num_objects":7537,"num_object_clones":0,"num_object_copies":15074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10074,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12413","reported_seq":12545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.359959+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:56.359959+0000","last_peered":"2024-09-11T05:18:56.359959+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:56.359959+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9446,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":1283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12079","reported_seq":12212,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.400425+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:56.400425+0000","last_peered":"2024-09-11T05:18:56.400425+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:56.400425+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31027200,"num_objects":7575,"num_object_clones":0,"num_object_copies":15150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8945,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4080},{"shards":"0,2","objects":1247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12200","reported_seq":12336,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.419664+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:56.419664+0000","last_peered":"2024-09-11T05:18:56.419664+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:56.419664+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8958,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12090","reported_seq":12234,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.470769+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:56.470769+0000","last_peered":"2024-09-11T05:18:56.470769+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:56.470769+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720617,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9097,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3986},{"shards":"0,2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":500789889,"num_objects":122265,"num_object_clones":0,"num_object_copies":244530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":682,"num_objects_unfound":0,"num_objects_dirty":122265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159430,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6399,"num_bytes_recovered":26193920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1122295808,"data_stored":1122214605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655928,"kb_used_data":359080,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93715912,"statfs":{"total":96636764160,"available":95965093888,"internally_reserved":0,"allocated":367697920,"data_stored":367438994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599388,"kb_used_data":315020,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93772452,"statfs":{"total":96636764160,"available":96022990848,"internally_reserved":0,"allocated":322580480,"data_stored":322254855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215181,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788200,"kb_used_data":423384,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93583640,"statfs":{"total":96636764160,"available":95829647360,"internally_reserved":0,"allocated":433545216,"data_stored":433286285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433111040,"data_stored":433107561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322199552,"data_stored":322129379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":366985216,"data_stored":366977665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:18:58.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:18:58.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12454", 'reported_seq': 12533, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.276041+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:18:56.276041+0000', 'last_peered': '2024-09-11T05:18:56.276041+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:18:56.276041+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31686656, 'num_objects': 7736, 'num_object_clones': 0, 'num_object_copies': 15472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12454, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4061}, {'shards': '0,2', 'objects': 1315}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:18:58.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12109", 'reported_seq': 12236, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.724924+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:18:56.724924+0000', 'last_peered': '2024-09-11T05:18:56.724924+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:18:56.724924+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31162392, 'num_objects': 7609, 'num_object_clones': 0, 'num_object_copies': 15218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7609, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10512, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1246}, {'shards': '2', 'objects': 4115}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:58.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:18:58.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12423", 'reported_seq': 18406, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.461864+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:18:56.461864+0000', 'last_peered': '2024-09-11T05:18:56.461864+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:18:56.461864+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31178752, 'num_objects': 7612, 'num_object_clones': 0, 'num_object_copies': 15224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 682, 'num_objects_unfound': 0, 'num_objects_dirty': 7612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8967, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2918, 'num_bytes_recovered': 11943936, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6928}, {'shards': '0,2', 'objects': 682}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:18:58.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12455", 'reported_seq': 12588, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.406059+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:18:56.406059+0000', 'last_peered': '2024-09-11T05:18:56.406059+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:18:56.406059+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31731712, 'num_objects': 7747, 'num_object_clones': 0, 'num_object_copies': 15494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9620, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4171}, {'shards': '0,2', 'objects': 1225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:18:58.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12319", 'reported_seq': 12450, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.749278+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:18:56.749278+0000', 'last_peered': '2024-09-11T05:18:56.749278+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:18:56.749278+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31342592, 'num_objects': 7652, 'num_object_clones': 0, 'num_object_copies': 15304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10387, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1195}, {'shards': '2', 'objects': 4127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:58.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:18:58.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12064", 'reported_seq': 12228, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.352599+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:18:56.352599+0000', 'last_peered': '2024-09-11T05:18:56.352599+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:18:56.352599+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31162368, 'num_objects': 7608, 'num_object_clones': 0, 'num_object_copies': 15216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8817, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4166}, {'shards': '0,2', 'objects': 1218}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12286", 'reported_seq': 12403, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.735782+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:18:56.735782+0000', 'last_peered': '2024-09-11T05:18:56.735782+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:18:56.735782+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31731712, 'num_objects': 7747, 'num_object_clones': 0, 'num_object_copies': 15494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10315, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1255}, {'shards': '2', 'objects': 4225}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12413", 'reported_seq': 12545, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.359959+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:18:56.359959+0000', 'last_peered': '2024-09-11T05:18:56.359959+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:18:56.359959+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31666176, 'num_objects': 7731, 'num_object_clones': 0, 'num_object_copies': 15462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9446, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4108}, {'shards': '0,2', 'objects': 1283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12079", 'reported_seq': 12212, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.400425+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:18:56.400425+0000', 'last_peered': '2024-09-11T05:18:56.400425+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:18:56.400425+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31027200, 'num_objects': 7575, 'num_object_clones': 0, 'num_object_copies': 15150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8945, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4080}, {'shards': '0,2', 'objects': 1247}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12200", 'reported_seq': 12336, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.419664+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:18:56.419664+0000', 'last_peered': '2024-09-11T05:18:56.419664+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:18:56.419664+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31129600, 'num_objects': 7600, 'num_object_clones': 0, 'num_object_copies': 15200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8958, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4056}, {'shards': '0,2', 'objects': 1245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:18:58.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12090", 'reported_seq': 12234, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:18:56.470769+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:18:56.470769+0000', 'last_peered': '2024-09-11T05:18:56.470769+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:18:56.470769+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30720617, 'num_objects': 7501, 'num_object_clones': 0, 'num_object_copies': 15002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9097, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3986}, {'shards': '0,2', 'objects': 1226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:18:58.324 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:18:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:18:59.336 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/244534 objects degraded (14.489%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":417,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122267,"data_bytes":500789908,"bytes_used":2092560384,"bytes_avail":287817732096,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":244534,"degraded_ratio":0.14489191687045563,"misplaced_objects":682,"misplaced_total":244534,"misplaced_ratio":0.0027889782197976559,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":115001,"recovering_keys_per_sec":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":559},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:18:59.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:18:59.403 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:00.349 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-11T05:18:59.532213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":500789908,"num_objects":122267,"num_object_clones":0,"num_object_copies":244534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":682,"num_objects_unfound":0,"num_objects_dirty":122267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159432,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6400,"num_bytes_recovered":26193939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2051104,"kb_used_data":1096752,"kb_used_omap":12,"kb_used_meta":954291,"kb_avail":281064416,"statfs":{"total":289910292480,"available":287809961984,"internally_reserved":0,"allocated":1123074048,"data_stored":1122230566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":977193985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,6,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18395136,"num_objects":-4491,"num_object_clones":0,"num_object_copies":-8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-218,"num_objects_unfound":0,"num_objects_dirty":-4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4491,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":892928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007851"},"pg_stats":[{"pgid":"2.f","version":"45'12454","reported_seq":12533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.276041+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:18:56.276041+0000","last_peered":"2024-09-11T05:18:56.276041+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:18:56.276041+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12454,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":1315}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12284","reported_seq":12387,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.625366+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:18:56.625366+0000","last_peered":"2024-09-11T05:18:56.625366+0000","last_clean":"2024-09-11T05:18:56.625366+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:18:56.625366+0000","last_undegraded":"2024-09-11T05:18:56.625366+0000","last_fullsized":"2024-09-11T05:18:56.625366+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10682,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12109","reported_seq":12236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.724924+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:18:56.724924+0000","last_peered":"2024-09-11T05:18:56.724924+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:18:56.724924+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162392,"num_objects":7609,"num_object_clones":0,"num_object_copies":15218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10512,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1246},{"shards":"2","objects":4115}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12348","reported_seq":12420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.678398+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:18:56.678398+0000","last_peered":"2024-09-11T05:18:56.678398+0000","last_clean":"2024-09-11T05:18:56.678398+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:18:56.678398+0000","last_undegraded":"2024-09-11T05:18:56.678398+0000","last_fullsized":"2024-09-11T05:18:56.678398+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31178752,"num_objects":7612,"num_object_clones":0,"num_object_copies":15224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12425","reported_seq":12555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.362129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:18:56.362129+0000","last_peered":"2024-09-11T05:18:56.362129+0000","last_clean":"2024-09-11T05:18:56.362129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:18:56.362129+0000","last_undegraded":"2024-09-11T05:18:56.362129+0000","last_fullsized":"2024-09-11T05:18:56.362129+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31719424,"num_objects":7744,"num_object_clones":0,"num_object_copies":15488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9259,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12423","reported_seq":18406,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.461864+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:18:56.461864+0000","last_peered":"2024-09-11T05:18:56.461864+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:18:56.461864+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31178752,"num_objects":7612,"num_object_clones":0,"num_object_copies":15224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":682,"num_objects_unfound":0,"num_objects_dirty":7612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8967,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2918,"num_bytes_recovered":11943936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6928},{"shards":"0,2","objects":682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12455","reported_seq":12588,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.406059+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:18:56.406059+0000","last_peered":"2024-09-11T05:18:56.406059+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:18:56.406059+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31731712,"num_objects":7747,"num_object_clones":0,"num_object_copies":15494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9620,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4171},{"shards":"0,2","objects":1225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12246","reported_seq":12371,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.259398+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:18:56.259398+0000","last_peered":"2024-09-11T05:18:56.259398+0000","last_clean":"2024-09-11T05:18:56.259398+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:18:56.259398+0000","last_undegraded":"2024-09-11T05:18:56.259398+0000","last_fullsized":"2024-09-11T05:18:56.259398+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12319","reported_seq":12450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.749278+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:18:56.749278+0000","last_peered":"2024-09-11T05:18:56.749278+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:18:56.749278+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31342592,"num_objects":7652,"num_object_clones":0,"num_object_copies":15304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10387,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1195},{"shards":"2","objects":4127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12064","reported_seq":12228,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.352599+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:18:56.352599+0000","last_peered":"2024-09-11T05:18:56.352599+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:18:56.352599+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8817,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4166},{"shards":"0,2","objects":1218}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12286","reported_seq":12403,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.735782+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:18:56.735782+0000","last_peered":"2024-09-11T05:18:56.735782+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:18:56.735782+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31731712,"num_objects":7747,"num_object_clones":0,"num_object_copies":15494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10315,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1255},{"shards":"2","objects":4225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12171","reported_seq":19257,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:18:56.695089+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:18:56.695089+0000","last_peered":"2024-09-11T05:18:56.695089+0000","last_clean":"2024-09-11T05:18:56.695089+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:18:56.695089+0000","last_undegraded":"2024-09-11T05:18:56.695089+0000","last_fullsized":"2024-09-11T05:18:56.695089+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30871552,"num_objects":7537,"num_object_clones":0,"num_object_copies":15074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10074,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12413","reported_seq":12545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.359959+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:18:56.359959+0000","last_peered":"2024-09-11T05:18:56.359959+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:18:56.359959+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9446,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":1283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12079","reported_seq":12212,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.400425+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:18:56.400425+0000","last_peered":"2024-09-11T05:18:56.400425+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:18:56.400425+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31027200,"num_objects":7575,"num_object_clones":0,"num_object_copies":15150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8945,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4080},{"shards":"0,2","objects":1247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12200","reported_seq":12336,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.419664+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:18:56.419664+0000","last_peered":"2024-09-11T05:18:56.419664+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:18:56.419664+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8958,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":1245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12090","reported_seq":12234,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:18:56.470769+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:18:56.470769+0000","last_peered":"2024-09-11T05:18:56.470769+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:18:56.470769+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720617,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9097,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3986},{"shards":"0,2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":500789889,"num_objects":122265,"num_object_clones":0,"num_object_copies":244530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":682,"num_objects_unfound":0,"num_objects_dirty":122265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159430,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6399,"num_bytes_recovered":26193920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1121468416,"data_stored":1121387213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655928,"kb_used_data":359080,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93715912,"statfs":{"total":96636764160,"available":95965093888,"internally_reserved":0,"allocated":367697920,"data_stored":367438994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606976,"kb_used_data":314288,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93764864,"statfs":{"total":96636764160,"available":96015220736,"internally_reserved":0,"allocated":321830912,"data_stored":321505287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215181,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788200,"kb_used_data":423384,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93583640,"statfs":{"total":96636764160,"available":95829647360,"internally_reserved":0,"allocated":433545216,"data_stored":433286285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433111040,"data_stored":433107561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321372160,"data_stored":321301987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":366985216,"data_stored":366977665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:03.425 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:04.254 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:04.312 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-11T05:19:03.533250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491606676,"num_objects":120025,"num_object_clones":0,"num_object_copies":240050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":120025,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161674,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6496,"num_bytes_recovered":26587155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2019144,"kb_used_data":1085588,"kb_used_omap":12,"kb_used_meta":933491,"kb_avail":281096376,"statfs":{"total":289910292480,"available":287842689024,"internally_reserved":0,"allocated":1111642112,"data_stored":1110798630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":955894785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,5,3,16,5,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21901312,"num_objects":-5347,"num_object_clones":0,"num_object_copies":-10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-204,"num_objects_unfound":0,"num_objects_dirty":-5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5347,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007877"},"pg_stats":[{"pgid":"2.f","version":"45'12594","reported_seq":12673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.501815+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:01.501815+0000","last_peered":"2024-09-11T05:19:01.501815+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:01.501815+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12594,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":1175}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12418","reported_seq":12521,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.745207+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:01.745207+0000","last_peered":"2024-09-11T05:19:01.745207+0000","last_clean":"2024-09-11T05:19:01.745207+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:01.745207+0000","last_undegraded":"2024-09-11T05:19:01.745207+0000","last_fullsized":"2024-09-11T05:19:01.745207+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30646272,"num_objects":7482,"num_object_clones":0,"num_object_copies":14964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10816,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12266","reported_seq":12393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.756221+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:01.756221+0000","last_peered":"2024-09-11T05:19:01.756221+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:01.756221+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519320,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10669,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1089},{"shards":"2","objects":3958}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12508","reported_seq":12580,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.771312+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:01.771312+0000","last_peered":"2024-09-11T05:19:01.771312+0000","last_clean":"2024-09-11T05:19:01.771312+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:01.771312+0000","last_undegraded":"2024-09-11T05:19:01.771312+0000","last_fullsized":"2024-09-11T05:19:01.771312+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12508,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12572","reported_seq":12702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.491630+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:01.491630+0000","last_peered":"2024-09-11T05:19:01.491630+0000","last_clean":"2024-09-11T05:19:01.491630+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:01.491630+0000","last_undegraded":"2024-09-11T05:19:01.491630+0000","last_fullsized":"2024-09-11T05:19:01.491630+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12562","reported_seq":18741,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.490788+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:01.490788+0000","last_peered":"2024-09-11T05:19:01.490788+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:01.490788+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30609408,"num_objects":7473,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":7473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9106,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3014,"num_bytes_recovered":12337152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6885},{"shards":"0,2","objects":586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12579","reported_seq":12712,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.438099+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:01.438099+0000","last_peered":"2024-09-11T05:19:01.438099+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:01.438099+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31223808,"num_objects":7623,"num_object_clones":0,"num_object_copies":15246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9744,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4047},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12375","reported_seq":12500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.456377+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:01.456377+0000","last_peered":"2024-09-11T05:19:01.456377+0000","last_clean":"2024-09-11T05:19:01.456377+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:01.456377+0000","last_undegraded":"2024-09-11T05:19:01.456377+0000","last_fullsized":"2024-09-11T05:19:01.456377+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9678,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12446","reported_seq":12578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.611441+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:01.611441+0000","last_peered":"2024-09-11T05:19:01.611441+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:01.611441+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30822400,"num_objects":7525,"num_object_clones":0,"num_object_copies":15050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10514,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1068},{"shards":"2","objects":4000}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12212","reported_seq":12378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.468962+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:01.468962+0000","last_peered":"2024-09-11T05:19:01.468962+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:01.468962+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30556160,"num_objects":7460,"num_object_clones":0,"num_object_copies":14920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8965,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4018},{"shards":"0,2","objects":1070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12417","reported_seq":12534,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.743559+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:01.743559+0000","last_peered":"2024-09-11T05:19:01.743559+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:01.743559+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10446,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1124},{"shards":"2","objects":4094}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12309","reported_seq":19395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.754680+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:01.754680+0000","last_peered":"2024-09-11T05:19:01.754680+0000","last_clean":"2024-09-11T05:19:01.754680+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:01.754680+0000","last_undegraded":"2024-09-11T05:19:01.754680+0000","last_fullsized":"2024-09-11T05:19:01.754680+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30306304,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10212,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12565","reported_seq":12697,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.401167+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:01.401167+0000","last_peered":"2024-09-11T05:19:01.401167+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:01.401167+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3956},{"shards":"0,2","objects":1131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12225","reported_seq":12359,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.454258+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:01.454258+0000","last_peered":"2024-09-11T05:19:01.454258+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:01.454258+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9091,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3934},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12321","reported_seq":12457,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.378185+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:01.378185+0000","last_peered":"2024-09-11T05:19:01.378185+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:01.378185+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30633984,"num_objects":7479,"num_object_clones":0,"num_object_copies":14958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9079,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3935},{"shards":"0,2","objects":1124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12239","reported_seq":12383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.446898+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:01.446898+0000","last_peered":"2024-09-11T05:19:01.446898+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:01.446898+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30110313,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":1077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491606657,"num_objects":120023,"num_object_clones":0,"num_object_copies":240046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":120023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161672,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6495,"num_bytes_recovered":26587136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1110032384,"data_stored":1109951181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639028,"kb_used_data":354656,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93732812,"statfs":{"total":96636764160,"available":95982399488,"internally_reserved":0,"allocated":363167744,"data_stored":362908818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606976,"kb_used_data":314288,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93764864,"statfs":{"total":96636764160,"available":96015220736,"internally_reserved":0,"allocated":321830912,"data_stored":321505287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215182,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773140,"kb_used_data":416644,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93598700,"statfs":{"total":96636764160,"available":95845068800,"internally_reserved":0,"allocated":426643456,"data_stored":426384525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,9,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426246144,"data_stored":426242665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321372160,"data_stored":321301987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362414080,"data_stored":362406529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:04.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:05.075 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:05.076 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:05.168 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-11T05:19:03.533250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491606676,"num_objects":120025,"num_object_clones":0,"num_object_copies":240050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":120025,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161674,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6496,"num_bytes_recovered":26587155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2019144,"kb_used_data":1085588,"kb_used_omap":12,"kb_used_meta":933491,"kb_avail":281096376,"statfs":{"total":289910292480,"available":287842689024,"internally_reserved":0,"allocated":1111642112,"data_stored":1110798630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":955894785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,5,3,16,5,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21901312,"num_objects":-5347,"num_object_clones":0,"num_object_copies":-10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-204,"num_objects_unfound":0,"num_objects_dirty":-5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5347,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007877"},"pg_stats":[{"pgid":"2.f","version":"45'12594","reported_seq":12673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.501815+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:01.501815+0000","last_peered":"2024-09-11T05:19:01.501815+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:01.501815+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12594,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":1175}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12418","reported_seq":12521,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.745207+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:01.745207+0000","last_peered":"2024-09-11T05:19:01.745207+0000","last_clean":"2024-09-11T05:19:01.745207+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:01.745207+0000","last_undegraded":"2024-09-11T05:19:01.745207+0000","last_fullsized":"2024-09-11T05:19:01.745207+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30646272,"num_objects":7482,"num_object_clones":0,"num_object_copies":14964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10816,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12266","reported_seq":12393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.756221+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:01.756221+0000","last_peered":"2024-09-11T05:19:01.756221+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:01.756221+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519320,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10669,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1089},{"shards":"2","objects":3958}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12508","reported_seq":12580,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.771312+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:01.771312+0000","last_peered":"2024-09-11T05:19:01.771312+0000","last_clean":"2024-09-11T05:19:01.771312+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:01.771312+0000","last_undegraded":"2024-09-11T05:19:01.771312+0000","last_fullsized":"2024-09-11T05:19:01.771312+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12508,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12572","reported_seq":12702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.491630+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:01.491630+0000","last_peered":"2024-09-11T05:19:01.491630+0000","last_clean":"2024-09-11T05:19:01.491630+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:01.491630+0000","last_undegraded":"2024-09-11T05:19:01.491630+0000","last_fullsized":"2024-09-11T05:19:01.491630+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12562","reported_seq":18741,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.490788+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:01.490788+0000","last_peered":"2024-09-11T05:19:01.490788+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:01.490788+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30609408,"num_objects":7473,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":7473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9106,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3014,"num_bytes_recovered":12337152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6885},{"shards":"0,2","objects":586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12579","reported_seq":12712,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.438099+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:01.438099+0000","last_peered":"2024-09-11T05:19:01.438099+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:01.438099+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31223808,"num_objects":7623,"num_object_clones":0,"num_object_copies":15246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9744,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4047},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12375","reported_seq":12500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.456377+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:01.456377+0000","last_peered":"2024-09-11T05:19:01.456377+0000","last_clean":"2024-09-11T05:19:01.456377+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:01.456377+0000","last_undegraded":"2024-09-11T05:19:01.456377+0000","last_fullsized":"2024-09-11T05:19:01.456377+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9678,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12446","reported_seq":12578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.611441+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:01.611441+0000","last_peered":"2024-09-11T05:19:01.611441+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:01.611441+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30822400,"num_objects":7525,"num_object_clones":0,"num_object_copies":15050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10514,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1068},{"shards":"2","objects":4000}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12212","reported_seq":12378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.468962+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:01.468962+0000","last_peered":"2024-09-11T05:19:01.468962+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:01.468962+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30556160,"num_objects":7460,"num_object_clones":0,"num_object_copies":14920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8965,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4018},{"shards":"0,2","objects":1070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12417","reported_seq":12534,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.743559+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:01.743559+0000","last_peered":"2024-09-11T05:19:01.743559+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:01.743559+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10446,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1124},{"shards":"2","objects":4094}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12309","reported_seq":19395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.754680+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:01.754680+0000","last_peered":"2024-09-11T05:19:01.754680+0000","last_clean":"2024-09-11T05:19:01.754680+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:01.754680+0000","last_undegraded":"2024-09-11T05:19:01.754680+0000","last_fullsized":"2024-09-11T05:19:01.754680+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30306304,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10212,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12565","reported_seq":12697,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.401167+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:01.401167+0000","last_peered":"2024-09-11T05:19:01.401167+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:01.401167+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3956},{"shards":"0,2","objects":1131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12225","reported_seq":12359,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.454258+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:01.454258+0000","last_peered":"2024-09-11T05:19:01.454258+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:01.454258+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9091,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3934},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12321","reported_seq":12457,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.378185+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:01.378185+0000","last_peered":"2024-09-11T05:19:01.378185+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:01.378185+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30633984,"num_objects":7479,"num_object_clones":0,"num_object_copies":14958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9079,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3935},{"shards":"0,2","objects":1124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12239","reported_seq":12383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.446898+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:01.446898+0000","last_peered":"2024-09-11T05:19:01.446898+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:01.446898+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30110313,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":1077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491606657,"num_objects":120023,"num_object_clones":0,"num_object_copies":240046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":120023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161672,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6495,"num_bytes_recovered":26587136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1110032384,"data_stored":1109951181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639028,"kb_used_data":354656,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93732812,"statfs":{"total":96636764160,"available":95982399488,"internally_reserved":0,"allocated":363167744,"data_stored":362908818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606976,"kb_used_data":314288,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93764864,"statfs":{"total":96636764160,"available":96015220736,"internally_reserved":0,"allocated":321830912,"data_stored":321505287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215182,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773140,"kb_used_data":416644,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93598700,"statfs":{"total":96636764160,"available":95845068800,"internally_reserved":0,"allocated":426643456,"data_stored":426384525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,9,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426246144,"data_stored":426242665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321372160,"data_stored":321301987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362414080,"data_stored":362406529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:05.169 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12594", 'reported_seq': 12673, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.501815+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:01.501815+0000', 'last_peered': '2024-09-11T05:19:01.501815+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:01.501815+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31113216, 'num_objects': 7596, 'num_object_clones': 0, 'num_object_copies': 15192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12594, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3921}, {'shards': '0,2', 'objects': 1175}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12266", 'reported_seq': 12393, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.756221+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:19:01.756221+0000', 'last_peered': '2024-09-11T05:19:01.756221+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:01.756221+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30519320, 'num_objects': 7452, 'num_object_clones': 0, 'num_object_copies': 14904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7452, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10669, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1089}, {'shards': '2', 'objects': 3958}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12562", 'reported_seq': 18741, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.490788+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:19:01.490788+0000', 'last_peered': '2024-09-11T05:19:01.490788+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:19:01.490788+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30609408, 'num_objects': 7473, 'num_object_clones': 0, 'num_object_copies': 14946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 586, 'num_objects_unfound': 0, 'num_objects_dirty': 7473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9106, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3014, 'num_bytes_recovered': 12337152, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6885}, {'shards': '0,2', 'objects': 586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12579", 'reported_seq': 12712, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.438099+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:01.438099+0000', 'last_peered': '2024-09-11T05:19:01.438099+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:01.438099+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31223808, 'num_objects': 7623, 'num_object_clones': 0, 'num_object_copies': 15246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9744, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4047}, {'shards': '0,2', 'objects': 1101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12446", 'reported_seq': 12578, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.611441+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:01.611441+0000', 'last_peered': '2024-09-11T05:19:01.611441+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:01.611441+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30822400, 'num_objects': 7525, 'num_object_clones': 0, 'num_object_copies': 15050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10514, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1068}, {'shards': '2', 'objects': 4000}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:05.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12212", 'reported_seq': 12378, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.468962+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:01.468962+0000', 'last_peered': '2024-09-11T05:19:01.468962+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:01.468962+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30556160, 'num_objects': 7460, 'num_object_clones': 0, 'num_object_copies': 14920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8965, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4018}, {'shards': '0,2', 'objects': 1070}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12417", 'reported_seq': 12534, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.743559+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:01.743559+0000', 'last_peered': '2024-09-11T05:19:01.743559+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:01.743559+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31195136, 'num_objects': 7616, 'num_object_clones': 0, 'num_object_copies': 15232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10446, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1124}, {'shards': '2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12565", 'reported_seq': 12697, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.401167+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:01.401167+0000', 'last_peered': '2024-09-11T05:19:01.401167+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:01.401167+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31043584, 'num_objects': 7579, 'num_object_clones': 0, 'num_object_copies': 15158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7579, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9598, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3956}, {'shards': '0,2', 'objects': 1131}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12225", 'reported_seq': 12359, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.454258+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:01.454258+0000', 'last_peered': '2024-09-11T05:19:01.454258+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:01.454258+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30429184, 'num_objects': 7429, 'num_object_clones': 0, 'num_object_copies': 14858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9091, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3934}, {'shards': '0,2', 'objects': 1101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12321", 'reported_seq': 12457, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.378185+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:01.378185+0000', 'last_peered': '2024-09-11T05:19:01.378185+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:01.378185+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30633984, 'num_objects': 7479, 'num_object_clones': 0, 'num_object_copies': 14958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9079, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3935}, {'shards': '0,2', 'objects': 1124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:05.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12239", 'reported_seq': 12383, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:01.446898+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:01.446898+0000', 'last_peered': '2024-09-11T05:19:01.446898+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:01.446898+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30110313, 'num_objects': 7352, 'num_object_clones': 0, 'num_object_copies': 14704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9246, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3837}, {'shards': '0,2', 'objects': 1077}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:05.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:06.185 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:06.186 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/240050 objects degraded (14.760%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":423,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120025,"data_bytes":491606676,"bytes_used":2054201344,"bytes_avail":287856091136,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":240050,"degraded_ratio":0.14759841699645906,"misplaced_objects":586,"misplaced_total":240050,"misplaced_ratio":0.0024411580920641533,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":69586,"recovering_keys_per_sec":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:06.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:06.226 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:07.092 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:07.092 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:07.139 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-11T05:19:05.533714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491606676,"num_objects":120025,"num_object_clones":0,"num_object_copies":240050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":120025,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161674,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6496,"num_bytes_recovered":26587155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006056,"kb_used_data":1084980,"kb_used_omap":12,"kb_used_meta":921011,"kb_avail":281109464,"statfs":{"total":289910292480,"available":287856091136,"internally_reserved":0,"allocated":1111019520,"data_stored":1110176038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":943115265},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,5,6,15,8,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18313216,"num_objects":-4471,"num_object_clones":0,"num_object_copies":-8942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-204,"num_objects_unfound":0,"num_objects_dirty":-4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007852"},"pg_stats":[{"pgid":"2.f","version":"45'12594","reported_seq":12673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.501815+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:01.501815+0000","last_peered":"2024-09-11T05:19:01.501815+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:01.501815+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12594,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":1175}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12418","reported_seq":12521,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.745207+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:01.745207+0000","last_peered":"2024-09-11T05:19:01.745207+0000","last_clean":"2024-09-11T05:19:01.745207+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:01.745207+0000","last_undegraded":"2024-09-11T05:19:01.745207+0000","last_fullsized":"2024-09-11T05:19:01.745207+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30646272,"num_objects":7482,"num_object_clones":0,"num_object_copies":14964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10816,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12266","reported_seq":12393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.756221+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:01.756221+0000","last_peered":"2024-09-11T05:19:01.756221+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:01.756221+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519320,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10669,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1089},{"shards":"2","objects":3958}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12508","reported_seq":12580,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.771312+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:01.771312+0000","last_peered":"2024-09-11T05:19:01.771312+0000","last_clean":"2024-09-11T05:19:01.771312+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:01.771312+0000","last_undegraded":"2024-09-11T05:19:01.771312+0000","last_fullsized":"2024-09-11T05:19:01.771312+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12508,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12572","reported_seq":12702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.491630+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:01.491630+0000","last_peered":"2024-09-11T05:19:01.491630+0000","last_clean":"2024-09-11T05:19:01.491630+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:01.491630+0000","last_undegraded":"2024-09-11T05:19:01.491630+0000","last_fullsized":"2024-09-11T05:19:01.491630+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12562","reported_seq":18741,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.490788+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:01.490788+0000","last_peered":"2024-09-11T05:19:01.490788+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:01.490788+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30609408,"num_objects":7473,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":7473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9106,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3014,"num_bytes_recovered":12337152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6885},{"shards":"0,2","objects":586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12579","reported_seq":12712,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.438099+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:01.438099+0000","last_peered":"2024-09-11T05:19:01.438099+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:01.438099+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31223808,"num_objects":7623,"num_object_clones":0,"num_object_copies":15246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9744,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4047},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12375","reported_seq":12500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.456377+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:01.456377+0000","last_peered":"2024-09-11T05:19:01.456377+0000","last_clean":"2024-09-11T05:19:01.456377+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:01.456377+0000","last_undegraded":"2024-09-11T05:19:01.456377+0000","last_fullsized":"2024-09-11T05:19:01.456377+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9678,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12446","reported_seq":12578,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.611441+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:01.611441+0000","last_peered":"2024-09-11T05:19:01.611441+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:01.611441+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30822400,"num_objects":7525,"num_object_clones":0,"num_object_copies":15050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10514,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1068},{"shards":"2","objects":4000}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12212","reported_seq":12378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.468962+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:01.468962+0000","last_peered":"2024-09-11T05:19:01.468962+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:01.468962+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30556160,"num_objects":7460,"num_object_clones":0,"num_object_copies":14920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8965,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4018},{"shards":"0,2","objects":1070}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12417","reported_seq":12534,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.743559+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:01.743559+0000","last_peered":"2024-09-11T05:19:01.743559+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:01.743559+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10446,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1124},{"shards":"2","objects":4094}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12309","reported_seq":19395,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:01.754680+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:01.754680+0000","last_peered":"2024-09-11T05:19:01.754680+0000","last_clean":"2024-09-11T05:19:01.754680+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:01.754680+0000","last_undegraded":"2024-09-11T05:19:01.754680+0000","last_fullsized":"2024-09-11T05:19:01.754680+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30306304,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10212,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12565","reported_seq":12697,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.401167+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:01.401167+0000","last_peered":"2024-09-11T05:19:01.401167+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:01.401167+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3956},{"shards":"0,2","objects":1131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12225","reported_seq":12359,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.454258+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:01.454258+0000","last_peered":"2024-09-11T05:19:01.454258+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:01.454258+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9091,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3934},{"shards":"0,2","objects":1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12321","reported_seq":12457,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.378185+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:01.378185+0000","last_peered":"2024-09-11T05:19:01.378185+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:01.378185+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30633984,"num_objects":7479,"num_object_clones":0,"num_object_copies":14958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9079,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3935},{"shards":"0,2","objects":1124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12239","reported_seq":12383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:01.446898+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:01.446898+0000","last_peered":"2024-09-11T05:19:01.446898+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:01.446898+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30110313,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":1077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491606657,"num_objects":120023,"num_object_clones":0,"num_object_copies":240046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":586,"num_objects_unfound":0,"num_objects_dirty":120023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161672,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6495,"num_bytes_recovered":26587136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1109409792,"data_stored":1109328589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639028,"kb_used_data":354656,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93732812,"statfs":{"total":96636764160,"available":95982399488,"internally_reserved":0,"allocated":363167744,"data_stored":362908818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593888,"kb_used_data":313680,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93777952,"statfs":{"total":96636764160,"available":96028622848,"internally_reserved":0,"allocated":321208320,"data_stored":320882695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215182,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773140,"kb_used_data":416644,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93598700,"statfs":{"total":96636764160,"available":95845068800,"internally_reserved":0,"allocated":426643456,"data_stored":426384525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,9,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426246144,"data_stored":426242665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320749568,"data_stored":320679395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362414080,"data_stored":362406529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:10.141 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:10.989 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:11.025 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-11T05:19:09.534721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":482579092,"num_objects":117821,"num_object_clones":0,"num_object_copies":235642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":470,"num_objects_unfound":0,"num_objects_dirty":117821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163878,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6612,"num_bytes_recovered":27062291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2024980,"kb_used_data":1073380,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":281090540,"statfs":{"total":289910292480,"available":287836712960,"internally_reserved":0,"allocated":1099141120,"data_stored":1098297638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":974375937},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,12,6,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18210816,"num_objects":-4446,"num_object_clones":0,"num_object_copies":-8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-212,"num_objects_unfound":0,"num_objects_dirty":-4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4446,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":868352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007903"},"pg_stats":[{"pgid":"2.f","version":"45'12760","reported_seq":12838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.455945+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:06.455945+0000","last_peered":"2024-09-11T05:19:06.455945+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:06.455945+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30433280,"num_objects":7430,"num_object_clones":0,"num_object_copies":14860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12760,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3755},{"shards":"0,2","objects":1009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12568","reported_seq":12671,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.791728+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:06.791728+0000","last_peered":"2024-09-11T05:19:06.791728+0000","last_clean":"2024-09-11T05:19:06.791728+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:06.791728+0000","last_undegraded":"2024-09-11T05:19:06.791728+0000","last_fullsized":"2024-09-11T05:19:06.791728+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12414","reported_seq":12541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.794364+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:06.794364+0000","last_peered":"2024-09-11T05:19:06.794364+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:06.794364+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29913112,"num_objects":7304,"num_object_clones":0,"num_object_copies":14608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7304,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10817,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":941},{"shards":"2","objects":3810}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12640","reported_seq":12712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.759689+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:06.759689+0000","last_peered":"2024-09-11T05:19:06.759689+0000","last_clean":"2024-09-11T05:19:06.759689+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:06.759689+0000","last_undegraded":"2024-09-11T05:19:06.759689+0000","last_fullsized":"2024-09-11T05:19:06.759689+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29982720,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12640,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12713","reported_seq":12842,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.423276+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:06.423276+0000","last_peered":"2024-09-11T05:19:06.423276+0000","last_clean":"2024-09-11T05:19:06.423276+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:06.423276+0000","last_undegraded":"2024-09-11T05:19:06.423276+0000","last_fullsized":"2024-09-11T05:19:06.423276+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30539776,"num_objects":7456,"num_object_clones":0,"num_object_copies":14912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9547,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12709","reported_seq":19120,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.529257+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:06.529257+0000","last_peered":"2024-09-11T05:19:06.529257+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:06.529257+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30007296,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":470,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3130,"num_bytes_recovered":12812288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6854},{"shards":"0,2","objects":470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12709","reported_seq":12842,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.390692+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:06.390692+0000","last_peered":"2024-09-11T05:19:06.390692+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:06.390692+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9874,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3917},{"shards":"0,2","objects":971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12480","reported_seq":12605,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.427137+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:06.427137+0000","last_peered":"2024-09-11T05:19:06.427137+0000","last_clean":"2024-09-11T05:19:06.427137+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:06.427137+0000","last_undegraded":"2024-09-11T05:19:06.427137+0000","last_fullsized":"2024-09-11T05:19:06.427137+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30326784,"num_objects":7404,"num_object_clones":0,"num_object_copies":14808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9783,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12562","reported_seq":12694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.597709+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:06.597709+0000","last_peered":"2024-09-11T05:19:06.597709+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:06.597709+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30347264,"num_objects":7409,"num_object_clones":0,"num_object_copies":14818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10630,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":3884}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12347","reported_seq":12513,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.511379+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:06.511379+0000","last_peered":"2024-09-11T05:19:06.511379+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:06.511379+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30003200,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9100,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3883},{"shards":"0,2","objects":935}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12559","reported_seq":12677,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.752467+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:06.752467+0000","last_peered":"2024-09-11T05:19:06.752467+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:06.752467+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613504,"num_objects":7474,"num_object_clones":0,"num_object_copies":14948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10588,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":982},{"shards":"2","objects":3952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12433","reported_seq":19519,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.749306+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:06.749306+0000","last_peered":"2024-09-11T05:19:06.749306+0000","last_clean":"2024-09-11T05:19:06.749306+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:06.749306+0000","last_undegraded":"2024-09-11T05:19:06.749306+0000","last_fullsized":"2024-09-11T05:19:06.749306+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29798400,"num_objects":7275,"num_object_clones":0,"num_object_copies":14550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12698","reported_seq":12830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.372090+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:06.372090+0000","last_peered":"2024-09-11T05:19:06.372090+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:06.372090+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3823},{"shards":"0,2","objects":998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12373","reported_seq":12503,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.500986+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:06.500986+0000","last_peered":"2024-09-11T05:19:06.500986+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:06.500986+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29822976,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9239,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12465","reported_seq":12601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.474715+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:06.474715+0000","last_peered":"2024-09-11T05:19:06.474715+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:06.474715+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30044160,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9223,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3791},{"shards":"0,2","objects":980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12382","reported_seq":12526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.450998+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:06.450998+0000","last_peered":"2024-09-11T05:19:06.450998+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:06.450998+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29524585,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9389,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3694},{"shards":"0,2","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":482579073,"num_objects":117819,"num_object_clones":0,"num_object_copies":235638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":470,"num_objects_unfound":0,"num_objects_dirty":117819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163876,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6611,"num_bytes_recovered":27062272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1097478144,"data_stored":1097396941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638892,"kb_used_data":350364,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93732948,"statfs":{"total":96636764160,"available":95982538752,"internally_reserved":0,"allocated":358772736,"data_stored":358513810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615428,"kb_used_data":313012,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93756412,"statfs":{"total":96636764160,"available":96006565888,"internally_reserved":0,"allocated":320524288,"data_stored":320198663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215183,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":770660,"kb_used_data":410004,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93601180,"statfs":{"total":96636764160,"available":95847608320,"internally_reserved":0,"allocated":419844096,"data_stored":419585165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419373056,"data_stored":419369577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320040960,"data_stored":319970787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":358064128,"data_stored":358056577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:11.027 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:11.731 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:11.807 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-11T05:19:11.539766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":482579092,"num_objects":117821,"num_object_clones":0,"num_object_copies":235642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":470,"num_objects_unfound":0,"num_objects_dirty":117821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163878,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6612,"num_bytes_recovered":27062291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2024980,"kb_used_data":1073380,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":281090540,"statfs":{"total":289910292480,"available":287836712960,"internally_reserved":0,"allocated":1099141120,"data_stored":1098297638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":974375937},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,12,6,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18210816,"num_objects":-4446,"num_object_clones":0,"num_object_copies":-8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-212,"num_objects_unfound":0,"num_objects_dirty":-4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4446,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":868352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007553"},"pg_stats":[{"pgid":"2.f","version":"45'12760","reported_seq":12838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.455945+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:06.455945+0000","last_peered":"2024-09-11T05:19:06.455945+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:06.455945+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30433280,"num_objects":7430,"num_object_clones":0,"num_object_copies":14860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12760,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3755},{"shards":"0,2","objects":1009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12568","reported_seq":12671,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.791728+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:06.791728+0000","last_peered":"2024-09-11T05:19:06.791728+0000","last_clean":"2024-09-11T05:19:06.791728+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:06.791728+0000","last_undegraded":"2024-09-11T05:19:06.791728+0000","last_fullsized":"2024-09-11T05:19:06.791728+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12414","reported_seq":12541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.794364+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:06.794364+0000","last_peered":"2024-09-11T05:19:06.794364+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:06.794364+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29913112,"num_objects":7304,"num_object_clones":0,"num_object_copies":14608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7304,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10817,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":941},{"shards":"2","objects":3810}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12640","reported_seq":12712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.759689+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:06.759689+0000","last_peered":"2024-09-11T05:19:06.759689+0000","last_clean":"2024-09-11T05:19:06.759689+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:06.759689+0000","last_undegraded":"2024-09-11T05:19:06.759689+0000","last_fullsized":"2024-09-11T05:19:06.759689+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29982720,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12640,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12713","reported_seq":12842,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.423276+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:06.423276+0000","last_peered":"2024-09-11T05:19:06.423276+0000","last_clean":"2024-09-11T05:19:06.423276+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:06.423276+0000","last_undegraded":"2024-09-11T05:19:06.423276+0000","last_fullsized":"2024-09-11T05:19:06.423276+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30539776,"num_objects":7456,"num_object_clones":0,"num_object_copies":14912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9547,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12709","reported_seq":19120,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.529257+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:06.529257+0000","last_peered":"2024-09-11T05:19:06.529257+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:06.529257+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30007296,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":470,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3130,"num_bytes_recovered":12812288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6854},{"shards":"0,2","objects":470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12709","reported_seq":12842,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.390692+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:06.390692+0000","last_peered":"2024-09-11T05:19:06.390692+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:06.390692+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9874,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3917},{"shards":"0,2","objects":971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12480","reported_seq":12605,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.427137+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:06.427137+0000","last_peered":"2024-09-11T05:19:06.427137+0000","last_clean":"2024-09-11T05:19:06.427137+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:06.427137+0000","last_undegraded":"2024-09-11T05:19:06.427137+0000","last_fullsized":"2024-09-11T05:19:06.427137+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30326784,"num_objects":7404,"num_object_clones":0,"num_object_copies":14808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9783,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12562","reported_seq":12694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.597709+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:06.597709+0000","last_peered":"2024-09-11T05:19:06.597709+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:06.597709+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30347264,"num_objects":7409,"num_object_clones":0,"num_object_copies":14818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10630,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":3884}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12347","reported_seq":12513,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.511379+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:06.511379+0000","last_peered":"2024-09-11T05:19:06.511379+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:06.511379+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30003200,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9100,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3883},{"shards":"0,2","objects":935}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":106,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.550304+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:15:57.550304+0000","last_peered":"2024-09-11T05:15:57.550304+0000","last_clean":"2024-09-11T05:15:57.550304+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:15:57.550304+0000","last_undegraded":"2024-09-11T05:15:57.550304+0000","last_fullsized":"2024-09-11T05:15:57.550304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12559","reported_seq":12677,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.752467+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:06.752467+0000","last_peered":"2024-09-11T05:19:06.752467+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:06.752467+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613504,"num_objects":7474,"num_object_clones":0,"num_object_copies":14948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10588,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":982},{"shards":"2","objects":3952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12433","reported_seq":19519,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:06.749306+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:06.749306+0000","last_peered":"2024-09-11T05:19:06.749306+0000","last_clean":"2024-09-11T05:19:06.749306+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:06.749306+0000","last_undegraded":"2024-09-11T05:19:06.749306+0000","last_fullsized":"2024-09-11T05:19:06.749306+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29798400,"num_objects":7275,"num_object_clones":0,"num_object_copies":14550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12698","reported_seq":12830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.372090+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:06.372090+0000","last_peered":"2024-09-11T05:19:06.372090+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:06.372090+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3823},{"shards":"0,2","objects":998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12373","reported_seq":12503,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.500986+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:06.500986+0000","last_peered":"2024-09-11T05:19:06.500986+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:06.500986+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29822976,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9239,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12465","reported_seq":12601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.474715+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:06.474715+0000","last_peered":"2024-09-11T05:19:06.474715+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:06.474715+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30044160,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9223,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3791},{"shards":"0,2","objects":980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12382","reported_seq":12526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:06.450998+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:06.450998+0000","last_peered":"2024-09-11T05:19:06.450998+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:06.450998+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29524585,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9389,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3694},{"shards":"0,2","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":482579073,"num_objects":117819,"num_object_clones":0,"num_object_copies":235638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":470,"num_objects_unfound":0,"num_objects_dirty":117819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163876,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6611,"num_bytes_recovered":27062272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1097478144,"data_stored":1097396941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638892,"kb_used_data":350364,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93732948,"statfs":{"total":96636764160,"available":95982538752,"internally_reserved":0,"allocated":358772736,"data_stored":358513810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615428,"kb_used_data":313012,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93756412,"statfs":{"total":96636764160,"available":96006565888,"internally_reserved":0,"allocated":320524288,"data_stored":320198663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215183,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":770660,"kb_used_data":410004,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93601180,"statfs":{"total":96636764160,"available":95847608320,"internally_reserved":0,"allocated":419844096,"data_stored":419585165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419373056,"data_stored":419369577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320040960,"data_stored":319970787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":358064128,"data_stored":358056577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:11.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:11.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12760", 'reported_seq': 12838, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.455945+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:06.455945+0000', 'last_peered': '2024-09-11T05:19:06.455945+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:06.455945+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30433280, 'num_objects': 7430, 'num_object_clones': 0, 'num_object_copies': 14860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12760, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3755}, {'shards': '0,2', 'objects': 1009}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:11.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12414", 'reported_seq': 12541, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.794364+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:19:06.794364+0000', 'last_peered': '2024-09-11T05:19:06.794364+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:06.794364+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29913112, 'num_objects': 7304, 'num_object_clones': 0, 'num_object_copies': 14608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7304, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10817, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 941}, {'shards': '2', 'objects': 3810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:11.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:19:11.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12709", 'reported_seq': 19120, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.529257+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:19:06.529257+0000', 'last_peered': '2024-09-11T05:19:06.529257+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:19:06.529257+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30007296, 'num_objects': 7326, 'num_object_clones': 0, 'num_object_copies': 14652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 470, 'num_objects_unfound': 0, 'num_objects_dirty': 7326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9253, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3130, 'num_bytes_recovered': 12812288, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6854}, {'shards': '0,2', 'objects': 470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:11.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12709", 'reported_seq': 12842, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.390692+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:06.390692+0000', 'last_peered': '2024-09-11T05:19:06.390692+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:06.390692+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30691328, 'num_objects': 7493, 'num_object_clones': 0, 'num_object_copies': 14986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9874, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3917}, {'shards': '0,2', 'objects': 971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:11.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12562", 'reported_seq': 12694, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.597709+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:06.597709+0000', 'last_peered': '2024-09-11T05:19:06.597709+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:06.597709+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30347264, 'num_objects': 7409, 'num_object_clones': 0, 'num_object_copies': 14818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10630, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 952}, {'shards': '2', 'objects': 3884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12347", 'reported_seq': 12513, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.511379+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:06.511379+0000', 'last_peered': '2024-09-11T05:19:06.511379+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:06.511379+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30003200, 'num_objects': 7325, 'num_object_clones': 0, 'num_object_copies': 14650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9100, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3883}, {'shards': '0,2', 'objects': 935}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12559", 'reported_seq': 12677, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.752467+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:06.752467+0000', 'last_peered': '2024-09-11T05:19:06.752467+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:06.752467+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30613504, 'num_objects': 7474, 'num_object_clones': 0, 'num_object_copies': 14948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10588, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 982}, {'shards': '2', 'objects': 3952}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12698", 'reported_seq': 12830, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.372090+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:06.372090+0000', 'last_peered': '2024-09-11T05:19:06.372090+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:06.372090+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30498816, 'num_objects': 7446, 'num_object_clones': 0, 'num_object_copies': 14892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9731, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3823}, {'shards': '0,2', 'objects': 998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12373", 'reported_seq': 12503, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.500986+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:06.500986+0000', 'last_peered': '2024-09-11T05:19:06.500986+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:06.500986+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29822976, 'num_objects': 7281, 'num_object_clones': 0, 'num_object_copies': 14562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7281, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9239, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3786}, {'shards': '0,2', 'objects': 953}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:11.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12465", 'reported_seq': 12601, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.474715+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:06.474715+0000', 'last_peered': '2024-09-11T05:19:06.474715+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:06.474715+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30044160, 'num_objects': 7335, 'num_object_clones': 0, 'num_object_copies': 14670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9223, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3791}, {'shards': '0,2', 'objects': 980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.812 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:11.812 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12382", 'reported_seq': 12526, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:06.450998+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:06.450998+0000', 'last_peered': '2024-09-11T05:19:06.450998+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:06.450998+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29524585, 'num_objects': 7209, 'num_object_clones': 0, 'num_object_copies': 14418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9389, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3694}, {'shards': '0,2', 'objects': 934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:11.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:12.831 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:12.832 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/235642 objects degraded (15.036%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":430,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":117821,"data_bytes":482579092,"bytes_used":2073579520,"bytes_avail":287836712960,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":235642,"degraded_ratio":0.15035944356269257,"misplaced_objects":470,"misplaced_total":235642,"misplaced_ratio":0.0019945510562633145,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":72317,"recovering_keys_per_sec":0,"num_objects_recovered":212,"num_bytes_recovered":868352,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":370},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:12.880 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:12.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:13.603 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:13.603 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:13.640 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-11T05:19:13.540206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":473223828,"num_objects":115537,"num_object_clones":0,"num_object_copies":231074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":344,"num_objects_unfound":0,"num_objects_dirty":115537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166162,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6738,"num_bytes_recovered":27578387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2025976,"kb_used_data":1061896,"kb_used_omap":12,"kb_used_meta":964019,"kb_avail":281089544,"statfs":{"total":289910292480,"available":287835693056,"internally_reserved":0,"allocated":1087381504,"data_stored":1086538022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":987155457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,12,6,9,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21852160,"num_objects":-5335,"num_object_clones":0,"num_object_copies":-10670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-242,"num_objects_unfound":0,"num_objects_dirty":-5335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5335,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":991232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007493"},"pg_stats":[{"pgid":"2.f","version":"45'12908","reported_seq":12987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.540546+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:11.540546+0000","last_peered":"2024-09-11T05:19:11.540546+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:11.540546+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12908,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":861}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12692","reported_seq":12795,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.672207+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:11.672207+0000","last_peered":"2024-09-11T05:19:11.672207+0000","last_clean":"2024-09-11T05:19:11.672207+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:11.672207+0000","last_undegraded":"2024-09-11T05:19:11.672207+0000","last_fullsized":"2024-09-11T05:19:11.672207+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523968,"num_objects":7208,"num_object_clones":0,"num_object_copies":14416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11090,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12548","reported_seq":12675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.815341+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:11.815341+0000","last_peered":"2024-09-11T05:19:11.815341+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:11.815341+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364248,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10951,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":807},{"shards":"2","objects":3676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12790","reported_seq":12861,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.835553+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:11.835553+0000","last_peered":"2024-09-11T05:19:11.835553+0000","last_clean":"2024-09-11T05:19:11.835553+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:11.835553+0000","last_undegraded":"2024-09-11T05:19:11.835553+0000","last_fullsized":"2024-09-11T05:19:11.835553+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368320,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12870","reported_seq":13000,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.505693+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:11.505693+0000","last_peered":"2024-09-11T05:19:11.505693+0000","last_clean":"2024-09-11T05:19:11.505693+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:11.505693+0000","last_undegraded":"2024-09-11T05:19:11.505693+0000","last_fullsized":"2024-09-11T05:19:11.505693+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29896704,"num_objects":7299,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9704,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12846","reported_seq":19509,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.531966+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:11.531966+0000","last_peered":"2024-09-11T05:19:11.531966+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:11.531966+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":344,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9390,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3256,"num_bytes_recovered":13328384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6843},{"shards":"0,2","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12842","reported_seq":12975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.470385+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:11.470385+0000","last_peered":"2024-09-11T05:19:11.470385+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:11.470385+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30146560,"num_objects":7360,"num_object_clones":0,"num_object_copies":14720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3784},{"shards":"0,2","objects":838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12626","reported_seq":12751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.411279+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:11.411279+0000","last_peered":"2024-09-11T05:19:11.411279+0000","last_clean":"2024-09-11T05:19:11.411279+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:11.411279+0000","last_undegraded":"2024-09-11T05:19:11.411279+0000","last_fullsized":"2024-09-11T05:19:11.411279+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29728768,"num_objects":7258,"num_object_clones":0,"num_object_copies":14516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12695","reported_seq":12824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.792310+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:11.792310+0000","last_peered":"2024-09-11T05:19:11.792310+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:11.792310+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29802496,"num_objects":7276,"num_object_clones":0,"num_object_copies":14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10763,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":819},{"shards":"2","objects":3751}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12481","reported_seq":12647,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.431601+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:11.431601+0000","last_peered":"2024-09-11T05:19:11.431601+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:11.431601+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29454336,"num_objects":7191,"num_object_clones":0,"num_object_copies":14382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9234,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.210646+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:17:05.210646+0000","last_peered":"2024-09-11T05:17:05.210646+0000","last_clean":"2024-09-11T05:17:05.210646+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:17:05.210646+0000","last_undegraded":"2024-09-11T05:17:05.210646+0000","last_fullsized":"2024-09-11T05:17:05.210646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12711","reported_seq":12828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.792368+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:11.792368+0000","last_peered":"2024-09-11T05:19:11.792368+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:11.792368+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":830},{"shards":"2","objects":3800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12578","reported_seq":19664,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.783673+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:11.783673+0000","last_peered":"2024-09-11T05:19:11.783673+0000","last_clean":"2024-09-11T05:19:11.783673+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:11.783673+0000","last_undegraded":"2024-09-11T05:19:11.783673+0000","last_fullsized":"2024-09-11T05:19:11.783673+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10481,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12847","reported_seq":12979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.539333+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:11.539333+0000","last_peered":"2024-09-11T05:19:11.539333+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:11.539333+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29888512,"num_objects":7297,"num_object_clones":0,"num_object_copies":14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3674},{"shards":"0,2","objects":849}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12507","reported_seq":12641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.292143+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:11.292143+0000","last_peered":"2024-09-11T05:19:11.292143+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:11.292143+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12619","reported_seq":12754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.520276+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:11.520276+0000","last_peered":"2024-09-11T05:19:11.520276+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:11.520276+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29413376,"num_objects":7181,"num_object_clones":0,"num_object_copies":14362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9377,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3637},{"shards":"0,2","objects":826}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12536","reported_seq":12680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.496279+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:11.496279+0000","last_peered":"2024-09-11T05:19:11.496279+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:11.496279+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28893801,"num_objects":7055,"num_object_clones":0,"num_object_copies":14110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9543,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3540},{"shards":"0,2","objects":780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":473223809,"num_objects":115535,"num_object_clones":0,"num_object_copies":231070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":344,"num_objects_unfound":0,"num_objects_dirty":115535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166160,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6737,"num_bytes_recovered":27578368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1085808640,"data_stored":1085727437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638500,"kb_used_data":345812,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93733340,"statfs":{"total":96636764160,"available":95982940160,"internally_reserved":0,"allocated":354111488,"data_stored":353852562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615428,"kb_used_data":313012,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93756412,"statfs":{"total":96636764160,"available":96006565888,"internally_reserved":0,"allocated":320524288,"data_stored":320198663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215184,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":772048,"kb_used_data":403072,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93599792,"statfs":{"total":96636764160,"available":95846187008,"internally_reserved":0,"allocated":412745728,"data_stored":412486797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":377810603},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":412352512,"data_stored":412349033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320040960,"data_stored":319970787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353415168,"data_stored":353407617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:16.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:17.531 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:17.532 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:17.602 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-11T05:19:15.540836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":473223828,"num_objects":115537,"num_object_clones":0,"num_object_copies":231074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":344,"num_objects_unfound":0,"num_objects_dirty":115537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166162,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6738,"num_bytes_recovered":27578387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2033716,"kb_used_data":1061316,"kb_used_omap":12,"kb_used_meta":972339,"kb_avail":281081804,"statfs":{"total":289910292480,"available":287827767296,"internally_reserved":0,"allocated":1086787584,"data_stored":1085944102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":995675137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,11,4,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18382848,"num_objects":-4488,"num_object_clones":0,"num_object_copies":-8976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-242,"num_objects_unfound":0,"num_objects_dirty":-4488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4488,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":991232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007586"},"pg_stats":[{"pgid":"2.f","version":"45'12908","reported_seq":12987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.540546+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:11.540546+0000","last_peered":"2024-09-11T05:19:11.540546+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:11.540546+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12908,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":861}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12692","reported_seq":12795,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.672207+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:11.672207+0000","last_peered":"2024-09-11T05:19:11.672207+0000","last_clean":"2024-09-11T05:19:11.672207+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:11.672207+0000","last_undegraded":"2024-09-11T05:19:11.672207+0000","last_fullsized":"2024-09-11T05:19:11.672207+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523968,"num_objects":7208,"num_object_clones":0,"num_object_copies":14416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11090,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12548","reported_seq":12675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.815341+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:11.815341+0000","last_peered":"2024-09-11T05:19:11.815341+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:11.815341+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364248,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10951,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":807},{"shards":"2","objects":3676}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12790","reported_seq":12861,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.835553+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:11.835553+0000","last_peered":"2024-09-11T05:19:11.835553+0000","last_clean":"2024-09-11T05:19:11.835553+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:11.835553+0000","last_undegraded":"2024-09-11T05:19:11.835553+0000","last_fullsized":"2024-09-11T05:19:11.835553+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368320,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12870","reported_seq":13000,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.505693+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:11.505693+0000","last_peered":"2024-09-11T05:19:11.505693+0000","last_clean":"2024-09-11T05:19:11.505693+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:11.505693+0000","last_undegraded":"2024-09-11T05:19:11.505693+0000","last_fullsized":"2024-09-11T05:19:11.505693+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29896704,"num_objects":7299,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9704,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12846","reported_seq":19509,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.531966+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:11.531966+0000","last_peered":"2024-09-11T05:19:11.531966+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:11.531966+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":344,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9390,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3256,"num_bytes_recovered":13328384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6843},{"shards":"0,2","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12842","reported_seq":12975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.470385+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:11.470385+0000","last_peered":"2024-09-11T05:19:11.470385+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:11.470385+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30146560,"num_objects":7360,"num_object_clones":0,"num_object_copies":14720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3784},{"shards":"0,2","objects":838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12626","reported_seq":12751,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.411279+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:11.411279+0000","last_peered":"2024-09-11T05:19:11.411279+0000","last_clean":"2024-09-11T05:19:11.411279+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:11.411279+0000","last_undegraded":"2024-09-11T05:19:11.411279+0000","last_fullsized":"2024-09-11T05:19:11.411279+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29728768,"num_objects":7258,"num_object_clones":0,"num_object_copies":14516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12695","reported_seq":12824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.792310+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:11.792310+0000","last_peered":"2024-09-11T05:19:11.792310+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:11.792310+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29802496,"num_objects":7276,"num_object_clones":0,"num_object_copies":14552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10763,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":819},{"shards":"2","objects":3751}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12481","reported_seq":12647,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.431601+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:11.431601+0000","last_peered":"2024-09-11T05:19:11.431601+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:11.431601+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29454336,"num_objects":7191,"num_object_clones":0,"num_object_copies":14382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9234,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.210646+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:17:05.210646+0000","last_peered":"2024-09-11T05:17:05.210646+0000","last_clean":"2024-09-11T05:17:05.210646+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:17:05.210646+0000","last_undegraded":"2024-09-11T05:17:05.210646+0000","last_fullsized":"2024-09-11T05:17:05.210646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12711","reported_seq":12828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.792368+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:11.792368+0000","last_peered":"2024-09-11T05:19:11.792368+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:11.792368+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":830},{"shards":"2","objects":3800}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12578","reported_seq":19664,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:11.783673+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:11.783673+0000","last_peered":"2024-09-11T05:19:11.783673+0000","last_clean":"2024-09-11T05:19:11.783673+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:11.783673+0000","last_undegraded":"2024-09-11T05:19:11.783673+0000","last_fullsized":"2024-09-11T05:19:11.783673+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10481,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12847","reported_seq":12979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.539333+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:11.539333+0000","last_peered":"2024-09-11T05:19:11.539333+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:11.539333+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29888512,"num_objects":7297,"num_object_clones":0,"num_object_copies":14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3674},{"shards":"0,2","objects":849}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12507","reported_seq":12641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.292143+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:11.292143+0000","last_peered":"2024-09-11T05:19:11.292143+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:11.292143+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12619","reported_seq":12754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.520276+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:11.520276+0000","last_peered":"2024-09-11T05:19:11.520276+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:11.520276+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29413376,"num_objects":7181,"num_object_clones":0,"num_object_copies":14362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9377,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3637},{"shards":"0,2","objects":826}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12536","reported_seq":12680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:11.496279+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:11.496279+0000","last_peered":"2024-09-11T05:19:11.496279+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:11.496279+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28893801,"num_objects":7055,"num_object_clones":0,"num_object_copies":14110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9543,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3540},{"shards":"0,2","objects":780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":473223809,"num_objects":115535,"num_object_clones":0,"num_object_copies":231070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":344,"num_objects_unfound":0,"num_objects_dirty":115535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166160,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6737,"num_bytes_recovered":27578368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1085296640,"data_stored":1085215437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638500,"kb_used_data":345812,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93733340,"statfs":{"total":96636764160,"available":95982940160,"internally_reserved":0,"allocated":354111488,"data_stored":353852562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623168,"kb_used_data":312432,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93748672,"statfs":{"total":96636764160,"available":95998640128,"internally_reserved":0,"allocated":319930368,"data_stored":319604743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215184,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":772048,"kb_used_data":403072,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93599792,"statfs":{"total":96636764160,"available":95846187008,"internally_reserved":0,"allocated":412745728,"data_stored":412486797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":377810603},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":412352512,"data_stored":412349033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319528960,"data_stored":319458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353415168,"data_stored":353407617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:17.603 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:18.445 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:18.483 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-11T05:19:17.541256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464110228,"num_objects":113312,"num_object_clones":0,"num_object_copies":226624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":206,"num_objects_unfound":0,"num_objects_dirty":113312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":168387,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6876,"num_bytes_recovered":28143635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998944,"kb_used_data":1051496,"kb_used_omap":12,"kb_used_meta":947379,"kb_avail":281116576,"statfs":{"total":289910292480,"available":287863373824,"internally_reserved":0,"allocated":1076731904,"data_stored":1075888422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":970116097},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,6,6,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27496448,"num_objects":-6713,"num_object_clones":0,"num_object_copies":-13426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":-6713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6713,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007542"},"pg_stats":[{"pgid":"2.f","version":"45'13071","reported_seq":13148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.579262+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:16.579262+0000","last_peered":"2024-09-11T05:19:16.579262+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:16.579262+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29159424,"num_objects":7119,"num_object_clones":0,"num_object_copies":14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13071,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3444},{"shards":"0,2","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12821","reported_seq":12924,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.777595+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:16.777595+0000","last_peered":"2024-09-11T05:19:16.777595+0000","last_clean":"2024-09-11T05:19:16.777595+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:16.777595+0000","last_undegraded":"2024-09-11T05:19:16.777595+0000","last_fullsized":"2024-09-11T05:19:16.777595+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28995584,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11219,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12674","reported_seq":12801,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.821921+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:16.821921+0000","last_peered":"2024-09-11T05:19:16.821921+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:16.821921+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28848152,"num_objects":7044,"num_object_clones":0,"num_object_copies":14088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7044,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11077,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":681},{"shards":"2","objects":3550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12928","reported_seq":13000,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.777996+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:16.777996+0000","last_peered":"2024-09-11T05:19:16.777996+0000","last_clean":"2024-09-11T05:19:16.777996+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:16.777996+0000","last_undegraded":"2024-09-11T05:19:16.777996+0000","last_fullsized":"2024-09-11T05:19:16.777996+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13011","reported_seq":13140,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.421884+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:16.421884+0000","last_peered":"2024-09-11T05:19:16.421884+0000","last_clean":"2024-09-11T05:19:16.421884+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:16.421884+0000","last_undegraded":"2024-09-11T05:19:16.421884+0000","last_fullsized":"2024-09-11T05:19:16.421884+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29319168,"num_objects":7158,"num_object_clones":0,"num_object_copies":14316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9845,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12966","reported_seq":19904,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.562804+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:16.562804+0000","last_peered":"2024-09-11T05:19:16.562804+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:16.562804+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":206,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9510,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3394,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6861},{"shards":"0,2","objects":206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12971","reported_seq":13104,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.519728+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:16.519728+0000","last_peered":"2024-09-11T05:19:16.519728+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:16.519728+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618176,"num_objects":7231,"num_object_clones":0,"num_object_copies":14462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10136,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12758","reported_seq":12883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.517095+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:16.517095+0000","last_peered":"2024-09-11T05:19:16.517095+0000","last_clean":"2024-09-11T05:19:16.517095+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:16.517095+0000","last_undegraded":"2024-09-11T05:19:16.517095+0000","last_fullsized":"2024-09-11T05:19:16.517095+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10061,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12847","reported_seq":12979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.799723+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:16.799723+0000","last_peered":"2024-09-11T05:19:16.799723+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:16.799723+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29179904,"num_objects":7124,"num_object_clones":0,"num_object_copies":14248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10915,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":667},{"shards":"2","objects":3599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12627","reported_seq":12793,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.482846+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:16.482846+0000","last_peered":"2024-09-11T05:19:16.482846+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:16.482846+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9380,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":655}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.210646+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:17:05.210646+0000","last_peered":"2024-09-11T05:17:05.210646+0000","last_clean":"2024-09-11T05:17:05.210646+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:17:05.210646+0000","last_undegraded":"2024-09-11T05:17:05.210646+0000","last_fullsized":"2024-09-11T05:17:05.210646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12875","reported_seq":12989,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.771539+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:16.771539+0000","last_peered":"2024-09-11T05:19:16.771539+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:16.771539+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29319168,"num_objects":7158,"num_object_clones":0,"num_object_copies":14316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10904,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":666},{"shards":"2","objects":3636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12693","reported_seq":19779,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.700485+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:16.700485+0000","last_peered":"2024-09-11T05:19:16.700485+0000","last_clean":"2024-09-11T05:19:16.700485+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:16.700485+0000","last_undegraded":"2024-09-11T05:19:16.700485+0000","last_fullsized":"2024-09-11T05:19:16.700485+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28733440,"num_objects":7015,"num_object_clones":0,"num_object_copies":14030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10596,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12992","reported_seq":13123,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.577273+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:16.577273+0000","last_peered":"2024-09-11T05:19:16.577273+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:16.577273+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29294592,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10025,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3529},{"shards":"0,2","objects":704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12647","reported_seq":12781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.545354+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:16.545354+0000","last_peered":"2024-09-11T05:19:16.545354+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:16.545354+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9513,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3512},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12752","reported_seq":12888,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.554222+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:16.554222+0000","last_peered":"2024-09-11T05:19:16.554222+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:16.554222+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28868608,"num_objects":7048,"num_object_clones":0,"num_object_copies":14096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9510,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3504},{"shards":"0,2","objects":693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12688","reported_seq":12832,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.494958+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:16.494958+0000","last_peered":"2024-09-11T05:19:16.494958+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:16.494958+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28271209,"num_objects":6903,"num_object_clones":0,"num_object_copies":13806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9695,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3388},{"shards":"0,2","objects":628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464110209,"num_objects":113310,"num_object_clones":0,"num_object_copies":226620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":206,"num_objects_unfound":0,"num_objects_dirty":113310,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":168385,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6875,"num_bytes_recovered":28143616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073901568,"data_stored":1073820365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160921,"ondisk_log_size":160921,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621612,"kb_used_data":341396,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93750228,"statfs":{"total":96636764160,"available":96000233472,"internally_reserved":0,"allocated":349589504,"data_stored":349330578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623168,"kb_used_data":312432,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93748672,"statfs":{"total":96636764160,"available":95998640128,"internally_reserved":0,"allocated":319930368,"data_stored":319604743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215185,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754164,"kb_used_data":397668,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93617676,"statfs":{"total":96636764160,"available":95864500224,"internally_reserved":0,"allocated":407212032,"data_stored":406953101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,2,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405454848,"data_stored":405451369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319528960,"data_stored":319458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348917760,"data_stored":348910209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:18.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:18.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13071", 'reported_seq': 13148, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.579262+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:16.579262+0000', 'last_peered': '2024-09-11T05:19:16.579262+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:16.579262+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29159424, 'num_objects': 7119, 'num_object_clones': 0, 'num_object_copies': 14238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13071, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3444}, {'shards': '0,2', 'objects': 698}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12674", 'reported_seq': 12801, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.821921+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:19:16.821921+0000', 'last_peered': '2024-09-11T05:19:16.821921+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:16.821921+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28848152, 'num_objects': 7044, 'num_object_clones': 0, 'num_object_copies': 14088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7044, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11077, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 681}, {'shards': '2', 'objects': 3550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12966", 'reported_seq': 19904, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.562804+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:19:16.562804+0000', 'last_peered': '2024-09-11T05:19:16.562804+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:19:16.562804+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28954624, 'num_objects': 7069, 'num_object_clones': 0, 'num_object_copies': 14138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 206, 'num_objects_unfound': 0, 'num_objects_dirty': 7069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9510, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3394, 'num_bytes_recovered': 13893632, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6861}, {'shards': '0,2', 'objects': 206}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12971", 'reported_seq': 13104, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.519728+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:16.519728+0000', 'last_peered': '2024-09-11T05:19:16.519728+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:16.519728+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29618176, 'num_objects': 7231, 'num_object_clones': 0, 'num_object_copies': 14462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10136, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3655}, {'shards': '0,2', 'objects': 709}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12847", 'reported_seq': 12979, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.799723+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:16.799723+0000', 'last_peered': '2024-09-11T05:19:16.799723+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:16.799723+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29179904, 'num_objects': 7124, 'num_object_clones': 0, 'num_object_copies': 14248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10915, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 667}, {'shards': '2', 'objects': 3599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12627", 'reported_seq': 12793, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.482846+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:16.482846+0000', 'last_peered': '2024-09-11T05:19:16.482846+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:16.482846+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28856320, 'num_objects': 7045, 'num_object_clones': 0, 'num_object_copies': 14090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9380, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3603}, {'shards': '0,2', 'objects': 655}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12875", 'reported_seq': 12989, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.771539+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:16.771539+0000', 'last_peered': '2024-09-11T05:19:16.771539+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:16.771539+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29319168, 'num_objects': 7158, 'num_object_clones': 0, 'num_object_copies': 14316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10904, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 666}, {'shards': '2', 'objects': 3636}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12992", 'reported_seq': 13123, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.577273+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:16.577273+0000', 'last_peered': '2024-09-11T05:19:16.577273+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:16.577273+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29294592, 'num_objects': 7152, 'num_object_clones': 0, 'num_object_copies': 14304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7152, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10025, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3529}, {'shards': '0,2', 'objects': 704}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12647", 'reported_seq': 12781, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.545354+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:16.545354+0000', 'last_peered': '2024-09-11T05:19:16.545354+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:16.545354+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28700672, 'num_objects': 7007, 'num_object_clones': 0, 'num_object_copies': 14014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9513, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3512}, {'shards': '0,2', 'objects': 679}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12752", 'reported_seq': 12888, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.554222+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:16.554222+0000', 'last_peered': '2024-09-11T05:19:16.554222+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:16.554222+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28868608, 'num_objects': 7048, 'num_object_clones': 0, 'num_object_copies': 14096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9510, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3504}, {'shards': '0,2', 'objects': 693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:18.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12688", 'reported_seq': 12832, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:16.494958+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:16.494958+0000', 'last_peered': '2024-09-11T05:19:16.494958+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:16.494958+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28271209, 'num_objects': 6903, 'num_object_clones': 0, 'num_object_copies': 13806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9695, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3388}, {'shards': '0,2', 'objects': 628}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:18.487 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:19.386 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:19.386 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/226624 objects degraded (15.634%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":437,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113312,"data_bytes":464110228,"bytes_used":2046918656,"bytes_avail":287863373824,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":226624,"degraded_ratio":0.15634266450155324,"misplaced_objects":206,"misplaced_total":226624,"misplaced_ratio":0.00090899463428410039,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":129625,"recovering_keys_per_sec":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":559},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:19.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:19.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:20.301 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:20.362 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-11T05:19:19.541749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464110228,"num_objects":113312,"num_object_clones":0,"num_object_copies":226624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":206,"num_objects_unfound":0,"num_objects_dirty":113312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":168387,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6876,"num_bytes_recovered":28143635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1990252,"kb_used_data":1051124,"kb_used_omap":12,"kb_used_meta":939059,"kb_avail":281125268,"statfs":{"total":289910292480,"available":287872274432,"internally_reserved":0,"allocated":1076350976,"data_stored":1075507494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":961596417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,5,6,14,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18468864,"num_objects":-4509,"num_object_clones":0,"num_object_copies":-9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":-4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4509,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007543"},"pg_stats":[{"pgid":"2.f","version":"45'13071","reported_seq":13148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.579262+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:16.579262+0000","last_peered":"2024-09-11T05:19:16.579262+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:16.579262+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29159424,"num_objects":7119,"num_object_clones":0,"num_object_copies":14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13071,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3444},{"shards":"0,2","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12821","reported_seq":12924,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.777595+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:16.777595+0000","last_peered":"2024-09-11T05:19:16.777595+0000","last_clean":"2024-09-11T05:19:16.777595+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:16.777595+0000","last_undegraded":"2024-09-11T05:19:16.777595+0000","last_fullsized":"2024-09-11T05:19:16.777595+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28995584,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11219,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12674","reported_seq":12801,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.821921+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:16.821921+0000","last_peered":"2024-09-11T05:19:16.821921+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:16.821921+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28848152,"num_objects":7044,"num_object_clones":0,"num_object_copies":14088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7044,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11077,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":681},{"shards":"2","objects":3550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12928","reported_seq":13000,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.777996+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:16.777996+0000","last_peered":"2024-09-11T05:19:16.777996+0000","last_clean":"2024-09-11T05:19:16.777996+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:16.777996+0000","last_undegraded":"2024-09-11T05:19:16.777996+0000","last_fullsized":"2024-09-11T05:19:16.777996+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13011","reported_seq":13140,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.421884+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:16.421884+0000","last_peered":"2024-09-11T05:19:16.421884+0000","last_clean":"2024-09-11T05:19:16.421884+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:16.421884+0000","last_undegraded":"2024-09-11T05:19:16.421884+0000","last_fullsized":"2024-09-11T05:19:16.421884+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29319168,"num_objects":7158,"num_object_clones":0,"num_object_copies":14316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9845,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12966","reported_seq":19904,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.562804+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:16.562804+0000","last_peered":"2024-09-11T05:19:16.562804+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:16.562804+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":206,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9510,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3394,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6861},{"shards":"0,2","objects":206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12971","reported_seq":13104,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.519728+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:16.519728+0000","last_peered":"2024-09-11T05:19:16.519728+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:16.519728+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618176,"num_objects":7231,"num_object_clones":0,"num_object_copies":14462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10136,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12758","reported_seq":12883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.517095+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:16.517095+0000","last_peered":"2024-09-11T05:19:16.517095+0000","last_clean":"2024-09-11T05:19:16.517095+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:16.517095+0000","last_undegraded":"2024-09-11T05:19:16.517095+0000","last_fullsized":"2024-09-11T05:19:16.517095+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10061,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12847","reported_seq":12979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.799723+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:16.799723+0000","last_peered":"2024-09-11T05:19:16.799723+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:16.799723+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29179904,"num_objects":7124,"num_object_clones":0,"num_object_copies":14248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10915,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":667},{"shards":"2","objects":3599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12627","reported_seq":12793,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.482846+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:16.482846+0000","last_peered":"2024-09-11T05:19:16.482846+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:16.482846+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9380,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":655}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.210646+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:17:05.210646+0000","last_peered":"2024-09-11T05:17:05.210646+0000","last_clean":"2024-09-11T05:17:05.210646+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:17:05.210646+0000","last_undegraded":"2024-09-11T05:17:05.210646+0000","last_fullsized":"2024-09-11T05:17:05.210646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12875","reported_seq":12989,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.771539+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:16.771539+0000","last_peered":"2024-09-11T05:19:16.771539+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:16.771539+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29319168,"num_objects":7158,"num_object_clones":0,"num_object_copies":14316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10904,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":666},{"shards":"2","objects":3636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12693","reported_seq":19779,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:16.700485+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:16.700485+0000","last_peered":"2024-09-11T05:19:16.700485+0000","last_clean":"2024-09-11T05:19:16.700485+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:16.700485+0000","last_undegraded":"2024-09-11T05:19:16.700485+0000","last_fullsized":"2024-09-11T05:19:16.700485+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28733440,"num_objects":7015,"num_object_clones":0,"num_object_copies":14030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10596,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12992","reported_seq":13123,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.577273+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:16.577273+0000","last_peered":"2024-09-11T05:19:16.577273+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:16.577273+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29294592,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10025,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3529},{"shards":"0,2","objects":704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12647","reported_seq":12781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.545354+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:16.545354+0000","last_peered":"2024-09-11T05:19:16.545354+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:16.545354+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9513,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3512},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12752","reported_seq":12888,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.554222+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:16.554222+0000","last_peered":"2024-09-11T05:19:16.554222+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:16.554222+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28868608,"num_objects":7048,"num_object_clones":0,"num_object_copies":14096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9510,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3504},{"shards":"0,2","objects":693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12688","reported_seq":12832,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:16.494958+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:16.494958+0000","last_peered":"2024-09-11T05:19:16.494958+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:16.494958+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28271209,"num_objects":6903,"num_object_clones":0,"num_object_copies":13806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9695,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3388},{"shards":"0,2","objects":628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464110209,"num_objects":113310,"num_object_clones":0,"num_object_copies":226620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":206,"num_objects_unfound":0,"num_objects_dirty":113310,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":168385,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6875,"num_bytes_recovered":28143616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073516544,"data_stored":1073435341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160921,"ondisk_log_size":160921,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621612,"kb_used_data":341396,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93750228,"statfs":{"total":96636764160,"available":96000233472,"internally_reserved":0,"allocated":349589504,"data_stored":349330578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614476,"kb_used_data":312060,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93757364,"statfs":{"total":96636764160,"available":96007540736,"internally_reserved":0,"allocated":319549440,"data_stored":319223815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215185,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754164,"kb_used_data":397668,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93617676,"statfs":{"total":96636764160,"available":95864500224,"internally_reserved":0,"allocated":407212032,"data_stored":406953101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,2,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405454848,"data_stored":405451369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319143936,"data_stored":319073763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348917760,"data_stored":348910209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:23.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:24.114 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:24.117 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:24.156 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-11T05:19:23.542663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":455090836,"num_objects":111110,"num_object_clones":0,"num_object_copies":222220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":111110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170589,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7017,"num_bytes_recovered":28721171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2005352,"kb_used_data":1039852,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281110168,"statfs":{"total":289910292480,"available":287856812032,"internally_reserved":0,"allocated":1064808448,"data_stored":1063964966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":988597249},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,9,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27488256,"num_objects":-6711,"num_object_clones":0,"num_object_copies":-13422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-405,"num_objects_unfound":0,"num_objects_dirty":-6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6711,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1658880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002896"},"pg_stats":[{"pgid":"2.f","version":"45'13202","reported_seq":13281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.567963+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:21.567963+0000","last_peered":"2024-09-11T05:19:21.567963+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:21.567963+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13202,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3313},{"shards":"0,2","objects":567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12973","reported_seq":13076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.787554+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:21.787554+0000","last_peered":"2024-09-11T05:19:21.787554+0000","last_clean":"2024-09-11T05:19:21.787554+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:21.787554+0000","last_undegraded":"2024-09-11T05:19:21.787554+0000","last_fullsized":"2024-09-11T05:19:21.787554+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28372992,"num_objects":6927,"num_object_clones":0,"num_object_copies":13854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11371,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12829","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.800336+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:21.800336+0000","last_peered":"2024-09-11T05:19:21.800336+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:21.800336+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28213272,"num_objects":6889,"num_object_clones":0,"num_object_copies":13778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11232,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":526},{"shards":"2","objects":3395}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13068","reported_seq":13138,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.862344+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:21.862344+0000","last_peered":"2024-09-11T05:19:21.862344+0000","last_clean":"2024-09-11T05:19:21.862344+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:21.862344+0000","last_undegraded":"2024-09-11T05:19:21.862344+0000","last_fullsized":"2024-09-11T05:19:21.862344+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28229632,"num_objects":6892,"num_object_clones":0,"num_object_copies":13784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13144","reported_seq":13274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.467543+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:21.467543+0000","last_peered":"2024-09-11T05:19:21.467543+0000","last_clean":"2024-09-11T05:19:21.467543+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:21.467543+0000","last_undegraded":"2024-09-11T05:19:21.467543+0000","last_fullsized":"2024-09-11T05:19:21.467543+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28774400,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9978,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13069","reported_seq":20290,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.608231+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:21.608231+0000","last_peered":"2024-09-11T05:19:21.608231+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:21.608231+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9613,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":14471168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6899},{"shards":"0,2","objects":65}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13123","reported_seq":13255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.598830+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:21.598830+0000","last_peered":"2024-09-11T05:19:21.598830+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:21.598830+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28995584,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10288,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12892","reported_seq":13017,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.565522+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:21.565522+0000","last_peered":"2024-09-11T05:19:21.565522+0000","last_clean":"2024-09-11T05:19:21.565522+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:21.565522+0000","last_undegraded":"2024-09-11T05:19:21.565522+0000","last_fullsized":"2024-09-11T05:19:21.565522+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28639232,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10195,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13002","reported_seq":13134,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.820340+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:21.820340+0000","last_peered":"2024-09-11T05:19:21.820340+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:21.820340+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11070,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":512},{"shards":"2","objects":3444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12754","reported_seq":12920,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.582145+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:21.582145+0000","last_peered":"2024-09-11T05:19:21.582145+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:21.582145+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28336128,"num_objects":6918,"num_object_clones":0,"num_object_copies":13836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9507,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.210646+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:17:05.210646+0000","last_peered":"2024-09-11T05:17:05.210646+0000","last_clean":"2024-09-11T05:17:05.210646+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:17:05.210646+0000","last_undegraded":"2024-09-11T05:17:05.210646+0000","last_fullsized":"2024-09-11T05:17:05.210646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13023","reported_seq":13141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.781924+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:21.781924+0000","last_peered":"2024-09-11T05:19:21.781924+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:21.781924+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28712960,"num_objects":7010,"num_object_clones":0,"num_object_copies":14020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11052,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":518},{"shards":"2","objects":3488}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12815","reported_seq":19901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.863078+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:21.863078+0000","last_peered":"2024-09-11T05:19:21.863078+0000","last_clean":"2024-09-11T05:19:21.863078+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:21.863078+0000","last_undegraded":"2024-09-11T05:19:21.863078+0000","last_fullsized":"2024-09-11T05:19:21.863078+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233728,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10718,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13141","reported_seq":13273,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.592798+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:21.592798+0000","last_peered":"2024-09-11T05:19:21.592798+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:21.592798+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684288,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10174,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3380},{"shards":"0,2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12772","reported_seq":12906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.490226+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:21.490226+0000","last_peered":"2024-09-11T05:19:21.490226+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:21.490226+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28188672,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9638,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12904","reported_seq":13040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.529074+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:21.529074+0000","last_peered":"2024-09-11T05:19:21.529074+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:21.529074+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28246016,"num_objects":6896,"num_object_clones":0,"num_object_copies":13792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3352},{"shards":"0,2","objects":541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12812","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.592007+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:21.592007+0000","last_peered":"2024-09-11T05:19:21.592007+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:21.592007+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27763305,"num_objects":6779,"num_object_clones":0,"num_object_copies":13558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9819,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3264},{"shards":"0,2","objects":504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":455090817,"num_objects":111108,"num_object_clones":0,"num_object_copies":222216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":111108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170587,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7016,"num_bytes_recovered":28721152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1062264832,"data_stored":1062183629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621164,"kb_used_data":336796,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93750676,"statfs":{"total":96636764160,"available":96000692224,"internally_reserved":0,"allocated":344879104,"data_stored":344620178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614476,"kb_used_data":312060,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93757364,"statfs":{"total":96636764160,"available":96007540736,"internally_reserved":0,"allocated":319549440,"data_stored":319223815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215186,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769712,"kb_used_data":390996,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93602128,"statfs":{"total":96636764160,"available":95848579072,"internally_reserved":0,"allocated":400379904,"data_stored":400120973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398843904,"data_stored":398840425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319143936,"data_stored":319073763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344276992,"data_stored":344269441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:24.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:24.932 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:24.933 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:24.971 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-11T05:19:24.682470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":455090836,"num_objects":111110,"num_object_clones":0,"num_object_copies":222220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":111110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170589,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7017,"num_bytes_recovered":28721171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009164,"kb_used_data":1039504,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281106356,"statfs":{"total":289910292480,"available":287852908544,"internally_reserved":0,"allocated":1064452096,"data_stored":1063608614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":992857089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,10,0,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18132992,"num_objects":-4427,"num_object_clones":0,"num_object_copies":-8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-279,"num_objects_unfound":0,"num_objects_dirty":-4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4427,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1142784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.142263"},"pg_stats":[{"pgid":"2.f","version":"45'13202","reported_seq":13281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.567963+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:21.567963+0000","last_peered":"2024-09-11T05:19:21.567963+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:21.567963+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13202,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3313},{"shards":"0,2","objects":567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12973","reported_seq":13076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.787554+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:21.787554+0000","last_peered":"2024-09-11T05:19:21.787554+0000","last_clean":"2024-09-11T05:19:21.787554+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:21.787554+0000","last_undegraded":"2024-09-11T05:19:21.787554+0000","last_fullsized":"2024-09-11T05:19:21.787554+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28372992,"num_objects":6927,"num_object_clones":0,"num_object_copies":13854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11371,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12829","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.800336+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:21.800336+0000","last_peered":"2024-09-11T05:19:21.800336+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:21.800336+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28213272,"num_objects":6889,"num_object_clones":0,"num_object_copies":13778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11232,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":526},{"shards":"2","objects":3395}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13068","reported_seq":13138,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.862344+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:21.862344+0000","last_peered":"2024-09-11T05:19:21.862344+0000","last_clean":"2024-09-11T05:19:21.862344+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:21.862344+0000","last_undegraded":"2024-09-11T05:19:21.862344+0000","last_fullsized":"2024-09-11T05:19:21.862344+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28229632,"num_objects":6892,"num_object_clones":0,"num_object_copies":13784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13144","reported_seq":13274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.467543+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:21.467543+0000","last_peered":"2024-09-11T05:19:21.467543+0000","last_clean":"2024-09-11T05:19:21.467543+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:21.467543+0000","last_undegraded":"2024-09-11T05:19:21.467543+0000","last_fullsized":"2024-09-11T05:19:21.467543+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28774400,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9978,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13069","reported_seq":20290,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.608231+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:21.608231+0000","last_peered":"2024-09-11T05:19:21.608231+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:21.608231+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9613,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":14471168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6899},{"shards":"0,2","objects":65}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13123","reported_seq":13255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.598830+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:21.598830+0000","last_peered":"2024-09-11T05:19:21.598830+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:21.598830+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28995584,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10288,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12892","reported_seq":13017,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.565522+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:21.565522+0000","last_peered":"2024-09-11T05:19:21.565522+0000","last_clean":"2024-09-11T05:19:21.565522+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:21.565522+0000","last_undegraded":"2024-09-11T05:19:21.565522+0000","last_fullsized":"2024-09-11T05:19:21.565522+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28639232,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10195,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13002","reported_seq":13134,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.820340+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:21.820340+0000","last_peered":"2024-09-11T05:19:21.820340+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:21.820340+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11070,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":512},{"shards":"2","objects":3444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12754","reported_seq":12920,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.582145+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:21.582145+0000","last_peered":"2024-09-11T05:19:21.582145+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:21.582145+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28336128,"num_objects":6918,"num_object_clones":0,"num_object_copies":13836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9507,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.210646+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:17:05.210646+0000","last_peered":"2024-09-11T05:17:05.210646+0000","last_clean":"2024-09-11T05:17:05.210646+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:17:05.210646+0000","last_undegraded":"2024-09-11T05:17:05.210646+0000","last_fullsized":"2024-09-11T05:17:05.210646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13023","reported_seq":13141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.781924+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:21.781924+0000","last_peered":"2024-09-11T05:19:21.781924+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:21.781924+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28712960,"num_objects":7010,"num_object_clones":0,"num_object_copies":14020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11052,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":518},{"shards":"2","objects":3488}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12815","reported_seq":19901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.863078+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:21.863078+0000","last_peered":"2024-09-11T05:19:21.863078+0000","last_clean":"2024-09-11T05:19:21.863078+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:21.863078+0000","last_undegraded":"2024-09-11T05:19:21.863078+0000","last_fullsized":"2024-09-11T05:19:21.863078+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233728,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10718,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13141","reported_seq":13273,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.592798+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:21.592798+0000","last_peered":"2024-09-11T05:19:21.592798+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:21.592798+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684288,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10174,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3380},{"shards":"0,2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12772","reported_seq":12906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.490226+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:21.490226+0000","last_peered":"2024-09-11T05:19:21.490226+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:21.490226+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28188672,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9638,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12904","reported_seq":13040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.529074+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:21.529074+0000","last_peered":"2024-09-11T05:19:21.529074+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:21.529074+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28246016,"num_objects":6896,"num_object_clones":0,"num_object_copies":13792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3352},{"shards":"0,2","objects":541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12812","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.592007+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:21.592007+0000","last_peered":"2024-09-11T05:19:21.592007+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:21.592007+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27763305,"num_objects":6779,"num_object_clones":0,"num_object_copies":13558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9819,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3264},{"shards":"0,2","objects":504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":455090817,"num_objects":111108,"num_object_clones":0,"num_object_copies":222216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":111108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170587,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7016,"num_bytes_recovered":28721152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1061818368,"data_stored":1061737165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621164,"kb_used_data":336796,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93750676,"statfs":{"total":96636764160,"available":96000692224,"internally_reserved":0,"allocated":344879104,"data_stored":344620178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618288,"kb_used_data":311712,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93753552,"statfs":{"total":96636764160,"available":96003637248,"internally_reserved":0,"allocated":319193088,"data_stored":318867463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,4,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215186,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769712,"kb_used_data":390996,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93602128,"statfs":{"total":96636764160,"available":95848579072,"internally_reserved":0,"allocated":400379904,"data_stored":400120973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398843904,"data_stored":398840425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318697472,"data_stored":318627299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344276992,"data_stored":344269441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13202", 'reported_seq': 13281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.567963+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:21.567963+0000', 'last_peered': '2024-09-11T05:19:21.567963+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:21.567963+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28622848, 'num_objects': 6988, 'num_object_clones': 0, 'num_object_copies': 13976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13202, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3313}, {'shards': '0,2', 'objects': 567}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12829", 'reported_seq': 12956, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.800336+0000', 'last_change': '2024-09-11T05:14:14.221560+0000', 'last_active': '2024-09-11T05:19:21.800336+0000', 'last_peered': '2024-09-11T05:19:21.800336+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:21.800336+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28213272, 'num_objects': 6889, 'num_object_clones': 0, 'num_object_copies': 13778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3495, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6889, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11232, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 526}, {'shards': '2', 'objects': 3395}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13069", 'reported_seq': 20290, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.608231+0000', 'last_change': '2024-09-11T05:15:56.251032+0000', 'last_active': '2024-09-11T05:19:21.608231+0000', 'last_peered': '2024-09-11T05:19:21.608231+0000', 'last_clean': '2024-09-11T05:14:12.019161+0000', 'last_became_active': '2024-09-11T05:14:14.193607+0000', 'last_became_peered': '2024-09-11T05:14:14.193607+0000', 'last_unstale': '2024-09-11T05:19:21.608231+0000', 'last_undegraded': '2024-09-11T05:14:13.902155+0000', 'last_fullsized': '2024-09-11T05:14:13.757960+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3536", 'last_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:11.134821+0000', 'objects_scrubbed': 3451, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1947, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28532736, 'num_objects': 6966, 'num_object_clones': 0, 'num_object_copies': 13932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 65, 'num_objects_unfound': 0, 'num_objects_dirty': 6966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9613, 'num_write_kb': 26256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3535, 'num_bytes_recovered': 14471168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6899}, {'shards': '0,2', 'objects': 65}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'13123", 'reported_seq': 13255, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.598830+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:21.598830+0000', 'last_peered': '2024-09-11T05:19:21.598830+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:21.598830+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28995584, 'num_objects': 7079, 'num_object_clones': 0, 'num_object_copies': 14158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7079, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10288, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3503}, {'shards': '0,2', 'objects': 557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:24.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13002", 'reported_seq': 13134, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.820340+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:21.820340+0000', 'last_peered': '2024-09-11T05:19:21.820340+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:21.820340+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28545024, 'num_objects': 6969, 'num_object_clones': 0, 'num_object_copies': 13938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6969, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11070, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 512}, {'shards': '2', 'objects': 3444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12754", 'reported_seq': 12920, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.582145+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:21.582145+0000', 'last_peered': '2024-09-11T05:19:21.582145+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:21.582145+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28336128, 'num_objects': 6918, 'num_object_clones': 0, 'num_object_copies': 13836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9507, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3476}, {'shards': '0,2', 'objects': 528}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'13023", 'reported_seq': 13141, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.781924+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:21.781924+0000', 'last_peered': '2024-09-11T05:19:21.781924+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:21.781924+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28712960, 'num_objects': 7010, 'num_object_clones': 0, 'num_object_copies': 14020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11052, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 518}, {'shards': '2', 'objects': 3488}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13141", 'reported_seq': 13273, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.592798+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:21.592798+0000', 'last_peered': '2024-09-11T05:19:21.592798+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:21.592798+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28684288, 'num_objects': 7003, 'num_object_clones': 0, 'num_object_copies': 14006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10174, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3380}, {'shards': '0,2', 'objects': 555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12772", 'reported_seq': 12906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.490226+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:21.490226+0000', 'last_peered': '2024-09-11T05:19:21.490226+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:21.490226+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28188672, 'num_objects': 6882, 'num_object_clones': 0, 'num_object_copies': 13764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9638, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3387}, {'shards': '0,2', 'objects': 554}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12904", 'reported_seq': 13040, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.529074+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:21.529074+0000', 'last_peered': '2024-09-11T05:19:21.529074+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:21.529074+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28246016, 'num_objects': 6896, 'num_object_clones': 0, 'num_object_copies': 13792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6896, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9662, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3352}, {'shards': '0,2', 'objects': 541}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:24.975 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12812", 'reported_seq': 12956, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:21.592007+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:21.592007+0000', 'last_peered': '2024-09-11T05:19:21.592007+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:21.592007+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27763305, 'num_objects': 6779, 'num_object_clones': 0, 'num_object_copies': 13558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9819, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3264}, {'shards': '0,2', 'objects': 504}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:24.975 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:25.562 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:25.562 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35431/222220 objects degraded (15.944%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111110,"data_bytes":455090836,"bytes_used":2053480448,"bytes_avail":287856812032,"bytes_total":289910292480,"degraded_objects":35431,"degraded_total":222220,"degraded_ratio":0.15944109441094412,"misplaced_objects":65,"misplaced_total":222220,"misplaced_ratio":0.00029250292502925029,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":138206,"recovering_keys_per_sec":0,"num_objects_recovered":405,"num_bytes_recovered":1658880,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":559},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:25.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:25.588 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:26.154 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:26.157 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:26.198 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-11T05:19:25.703340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":455090836,"num_objects":111110,"num_object_clones":0,"num_object_copies":222220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":111110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170589,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7017,"num_bytes_recovered":28721171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009164,"kb_used_data":1039504,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281106356,"statfs":{"total":289910292480,"available":287852908544,"internally_reserved":0,"allocated":1064452096,"data_stored":1063608614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":992857089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,10,0,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9019392,"num_objects":-2202,"num_object_clones":0,"num_object_copies":-4404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-141,"num_objects_unfound":0,"num_objects_dirty":-2202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2202,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":577536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.162084"},"pg_stats":[{"pgid":"2.f","version":"45'13202","reported_seq":13281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.567963+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:21.567963+0000","last_peered":"2024-09-11T05:19:21.567963+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:21.567963+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13202,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3313},{"shards":"0,2","objects":567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12973","reported_seq":13076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.787554+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:21.787554+0000","last_peered":"2024-09-11T05:19:21.787554+0000","last_clean":"2024-09-11T05:19:21.787554+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:21.787554+0000","last_undegraded":"2024-09-11T05:19:21.787554+0000","last_fullsized":"2024-09-11T05:19:21.787554+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28372992,"num_objects":6927,"num_object_clones":0,"num_object_copies":13854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11371,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12829","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.800336+0000","last_change":"2024-09-11T05:14:14.221560+0000","last_active":"2024-09-11T05:19:21.800336+0000","last_peered":"2024-09-11T05:19:21.800336+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:21.800336+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28213272,"num_objects":6889,"num_object_clones":0,"num_object_copies":13778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11232,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":526},{"shards":"2","objects":3395}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13068","reported_seq":13138,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.862344+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:21.862344+0000","last_peered":"2024-09-11T05:19:21.862344+0000","last_clean":"2024-09-11T05:19:21.862344+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:21.862344+0000","last_undegraded":"2024-09-11T05:19:21.862344+0000","last_fullsized":"2024-09-11T05:19:21.862344+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28229632,"num_objects":6892,"num_object_clones":0,"num_object_copies":13784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13144","reported_seq":13274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.467543+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:21.467543+0000","last_peered":"2024-09-11T05:19:21.467543+0000","last_clean":"2024-09-11T05:19:21.467543+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:21.467543+0000","last_undegraded":"2024-09-11T05:19:21.467543+0000","last_fullsized":"2024-09-11T05:19:21.467543+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28774400,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9978,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13069","reported_seq":20290,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.608231+0000","last_change":"2024-09-11T05:15:56.251032+0000","last_active":"2024-09-11T05:19:21.608231+0000","last_peered":"2024-09-11T05:19:21.608231+0000","last_clean":"2024-09-11T05:14:12.019161+0000","last_became_active":"2024-09-11T05:14:14.193607+0000","last_became_peered":"2024-09-11T05:14:14.193607+0000","last_unstale":"2024-09-11T05:19:21.608231+0000","last_undegraded":"2024-09-11T05:14:13.902155+0000","last_fullsized":"2024-09-11T05:14:13.757960+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9613,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3535,"num_bytes_recovered":14471168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6899},{"shards":"0,2","objects":65}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13123","reported_seq":13255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.598830+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:21.598830+0000","last_peered":"2024-09-11T05:19:21.598830+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:21.598830+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28995584,"num_objects":7079,"num_object_clones":0,"num_object_copies":14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10288,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":557}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12892","reported_seq":13017,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.565522+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:21.565522+0000","last_peered":"2024-09-11T05:19:21.565522+0000","last_clean":"2024-09-11T05:19:21.565522+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:21.565522+0000","last_undegraded":"2024-09-11T05:19:21.565522+0000","last_fullsized":"2024-09-11T05:19:21.565522+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28639232,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10195,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:11.695761+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:16:11.695761+0000","last_peered":"2024-09-11T05:16:11.695761+0000","last_clean":"2024-09-11T05:16:11.695761+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:16:11.695761+0000","last_undegraded":"2024-09-11T05:16:11.695761+0000","last_fullsized":"2024-09-11T05:16:11.695761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13002","reported_seq":13134,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.820340+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:21.820340+0000","last_peered":"2024-09-11T05:19:21.820340+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:21.820340+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11070,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":512},{"shards":"2","objects":3444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.566739+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:15:57.566739+0000","last_peered":"2024-09-11T05:15:57.566739+0000","last_clean":"2024-09-11T05:15:57.566739+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:15:57.566739+0000","last_undegraded":"2024-09-11T05:15:57.566739+0000","last_fullsized":"2024-09-11T05:15:57.566739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12754","reported_seq":12920,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.582145+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:21.582145+0000","last_peered":"2024-09-11T05:19:21.582145+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:21.582145+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28336128,"num_objects":6918,"num_object_clones":0,"num_object_copies":13836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9507,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":107,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:17:05.210646+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:17:05.210646+0000","last_peered":"2024-09-11T05:17:05.210646+0000","last_clean":"2024-09-11T05:17:05.210646+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:17:05.210646+0000","last_undegraded":"2024-09-11T05:17:05.210646+0000","last_fullsized":"2024-09-11T05:17:05.210646+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13023","reported_seq":13141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.781924+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:21.781924+0000","last_peered":"2024-09-11T05:19:21.781924+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:21.781924+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28712960,"num_objects":7010,"num_object_clones":0,"num_object_copies":14020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11052,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":518},{"shards":"2","objects":3488}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834134+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:16:16.834134+0000","last_peered":"2024-09-11T05:16:16.834134+0000","last_clean":"2024-09-11T05:16:16.834134+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:16:16.834134+0000","last_undegraded":"2024-09-11T05:16:16.834134+0000","last_fullsized":"2024-09-11T05:16:16.834134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12815","reported_seq":19901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:19:21.863078+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:21.863078+0000","last_peered":"2024-09-11T05:19:21.863078+0000","last_clean":"2024-09-11T05:19:21.863078+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:21.863078+0000","last_undegraded":"2024-09-11T05:19:21.863078+0000","last_fullsized":"2024-09-11T05:19:21.863078+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233728,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10718,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13141","reported_seq":13273,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.592798+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:21.592798+0000","last_peered":"2024-09-11T05:19:21.592798+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:21.592798+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28684288,"num_objects":7003,"num_object_clones":0,"num_object_copies":14006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10174,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3380},{"shards":"0,2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834169+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:16:16.834169+0000","last_peered":"2024-09-11T05:16:16.834169+0000","last_clean":"2024-09-11T05:16:16.834169+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:16:16.834169+0000","last_undegraded":"2024-09-11T05:16:16.834169+0000","last_fullsized":"2024-09-11T05:16:16.834169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12772","reported_seq":12906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.490226+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:21.490226+0000","last_peered":"2024-09-11T05:19:21.490226+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:21.490226+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28188672,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9638,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:15:57.567271+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:15:57.567271+0000","last_peered":"2024-09-11T05:15:57.567271+0000","last_clean":"2024-09-11T05:15:57.567271+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:15:57.567271+0000","last_undegraded":"2024-09-11T05:15:57.567271+0000","last_fullsized":"2024-09-11T05:15:57.567271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12904","reported_seq":13040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.529074+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:21.529074+0000","last_peered":"2024-09-11T05:19:21.529074+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:21.529074+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28246016,"num_objects":6896,"num_object_clones":0,"num_object_copies":13792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3352},{"shards":"0,2","objects":541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:16.834180+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:16:16.834180+0000","last_peered":"2024-09-11T05:16:16.834180+0000","last_clean":"2024-09-11T05:16:16.834180+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:16:16.834180+0000","last_undegraded":"2024-09-11T05:16:16.834180+0000","last_fullsized":"2024-09-11T05:16:16.834180+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T05:16:21.883528+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:16:21.883528+0000","last_peered":"2024-09-11T05:16:21.883528+0000","last_clean":"2024-09-11T05:16:21.883528+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:16:21.883528+0000","last_undegraded":"2024-09-11T05:16:21.883528+0000","last_fullsized":"2024-09-11T05:16:21.883528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12812","reported_seq":12956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:21.592007+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:21.592007+0000","last_peered":"2024-09-11T05:19:21.592007+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:21.592007+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27763305,"num_objects":6779,"num_object_clones":0,"num_object_copies":13558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9819,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3264},{"shards":"0,2","objects":504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":455090817,"num_objects":111108,"num_object_clones":0,"num_object_copies":222216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35431,"num_objects_misplaced":65,"num_objects_unfound":0,"num_objects_dirty":111108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170587,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7016,"num_bytes_recovered":28721152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1061818368,"data_stored":1061737165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621164,"kb_used_data":336796,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93750676,"statfs":{"total":96636764160,"available":96000692224,"internally_reserved":0,"allocated":344879104,"data_stored":344620178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618288,"kb_used_data":311712,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93753552,"statfs":{"total":96636764160,"available":96003637248,"internally_reserved":0,"allocated":319193088,"data_stored":318867463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,4,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215186,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769712,"kb_used_data":390996,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93602128,"statfs":{"total":96636764160,"available":95848579072,"internally_reserved":0,"allocated":400379904,"data_stored":400120973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398843904,"data_stored":398840425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318697472,"data_stored":318627299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344276992,"data_stored":344269441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:29.200 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:29.935 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:29.986 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-11T05:19:29.545866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":446235284,"num_objects":108948,"num_object_clones":0,"num_object_copies":217896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34988,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":108948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172751,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7150,"num_bytes_recovered":29261843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010388,"kb_used_data":1028244,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281105132,"statfs":{"total":289910292480,"available":287851655168,"internally_reserved":0,"allocated":1052921856,"data_stored":1052050518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1005636609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,3,6,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17874944,"num_objects":-4364,"num_object_clones":0,"num_object_copies":-8728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-443,"num_objects_misplaced":169,"num_objects_unfound":0,"num_objects_dirty":-4364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4364,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003642"},"pg_stats":[{"pgid":"2.f","version":"47'13335","reported_seq":13417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.583540+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:26.583540+0000","last_peered":"2024-09-11T05:19:26.583540+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:26.583540+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28078080,"num_objects":6855,"num_object_clones":0,"num_object_copies":13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3180},{"shards":"0,2","objects":434}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13086","reported_seq":13193,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.831002+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:26.831002+0000","last_peered":"2024-09-11T05:19:26.831002+0000","last_clean":"2024-09-11T05:19:26.831002+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:26.831002+0000","last_undegraded":"2024-09-11T05:19:26.831002+0000","last_fullsized":"2024-09-11T05:19:26.831002+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27910144,"num_objects":6814,"num_object_clones":0,"num_object_copies":13628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11484,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12966","reported_seq":13227,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.889643+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:26.889643+0000","last_peered":"2024-09-11T05:19:26.889643+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:26.889643+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27652120,"num_objects":6752,"num_object_clones":0,"num_object_copies":13504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3054,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":6752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11369,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":375},{"shards":"2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13193","reported_seq":13269,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.846407+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:26.846407+0000","last_peered":"2024-09-11T05:19:26.846407+0000","last_clean":"2024-09-11T05:19:26.846407+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:26.846407+0000","last_undegraded":"2024-09-11T05:19:26.846407+0000","last_fullsized":"2024-09-11T05:19:26.846407+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13193,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13283","reported_seq":13417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.512306+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:26.512306+0000","last_peered":"2024-09-11T05:19:26.512306+0000","last_clean":"2024-09-11T05:19:26.512306+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:26.512306+0000","last_undegraded":"2024-09-11T05:19:26.512306+0000","last_fullsized":"2024-09-11T05:19:26.512306+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10117,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13254","reported_seq":20605,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:28.872505+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:28.872505+0000","last_peered":"2024-09-11T05:19:28.872505+0000","last_clean":"2024-09-11T05:19:28.872505+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:28.872505+0000","last_undegraded":"2024-09-11T05:19:28.872505+0000","last_fullsized":"2024-09-11T05:19:28.872505+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13259","reported_seq":13396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.550572+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:26.550572+0000","last_peered":"2024-09-11T05:19:26.550572+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:26.550572+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28438528,"num_objects":6943,"num_object_clones":0,"num_object_copies":13886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10424,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3367},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13033","reported_seq":13162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.475003+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:26.475003+0000","last_peered":"2024-09-11T05:19:26.475003+0000","last_clean":"2024-09-11T05:19:26.475003+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:26.475003+0000","last_undegraded":"2024-09-11T05:19:26.475003+0000","last_fullsized":"2024-09-11T05:19:26.475003+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28061696,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13132","reported_seq":13268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.870663+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:26.870663+0000","last_peered":"2024-09-11T05:19:26.870663+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:26.870663+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":382},{"shards":"2","objects":3314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704881+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704881+0000","last_peered":"2024-09-11T05:19:25.704881+0000","last_clean":"2024-09-11T05:19:25.704881+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704881+0000","last_undegraded":"2024-09-11T05:19:25.704881+0000","last_fullsized":"2024-09-11T05:19:25.704881+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12903","reported_seq":13073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.583583+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:26.583583+0000","last_peered":"2024-09-11T05:19:26.583583+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:26.583583+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9656,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3327},{"shards":"0,2","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13144","reported_seq":13264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.837021+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:26.837021+0000","last_peered":"2024-09-11T05:19:26.837021+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:26.837021+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28217344,"num_objects":6889,"num_object_clones":0,"num_object_copies":13778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11173,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":397},{"shards":"2","objects":3367}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12940","reported_seq":20030,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.839723+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:26.839723+0000","last_peered":"2024-09-11T05:19:26.839723+0000","last_clean":"2024-09-11T05:19:26.839723+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:26.839723+0000","last_undegraded":"2024-09-11T05:19:26.839723+0000","last_fullsized":"2024-09-11T05:19:26.839723+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":6768,"num_object_clones":0,"num_object_copies":13536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10843,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13276","reported_seq":13407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.547483+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:26.547483+0000","last_peered":"2024-09-11T05:19:26.547483+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:26.547483+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3245},{"shards":"0,2","objects":420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12909","reported_seq":13048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.594159+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:26.594159+0000","last_peered":"2024-09-11T05:19:26.594159+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:26.594159+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27627520,"num_objects":6745,"num_object_clones":0,"num_object_copies":13490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9775,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3250},{"shards":"0,2","objects":417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.706987+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.706987+0000","last_peered":"2024-09-11T05:19:25.706987+0000","last_clean":"2024-09-11T05:19:25.706987+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.706987+0000","last_undegraded":"2024-09-11T05:19:25.706987+0000","last_fullsized":"2024-09-11T05:19:25.706987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13040","reported_seq":13180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.490735+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:26.490735+0000","last_peered":"2024-09-11T05:19:26.490735+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:26.490735+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27688960,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3216},{"shards":"0,2","objects":405}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12932","reported_seq":13080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.531906+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:26.531906+0000","last_peered":"2024-09-11T05:19:26.531906+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:26.531906+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27271785,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9939,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":446235265,"num_objects":108946,"num_object_clones":0,"num_object_copies":217892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34988,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":108946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172749,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7149,"num_bytes_recovered":29261824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050132480,"data_stored":1050054756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621232,"kb_used_data":332700,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93750608,"statfs":{"total":96636764160,"available":96000622592,"internally_reserved":0,"allocated":340684800,"data_stored":340415429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621596,"kb_used_data":310856,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750244,"statfs":{"total":96636764160,"available":96000249856,"internally_reserved":0,"allocated":318316544,"data_stored":317983953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215187,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767560,"kb_used_data":384688,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93604280,"statfs":{"total":96636764160,"available":95850782720,"internally_reserved":0,"allocated":393920512,"data_stored":393651136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":392384512,"data_stored":392381033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317706240,"data_stored":317639546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340041728,"data_stored":340034177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:29.987 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:30.858 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:30.859 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:30.963 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-11T05:19:29.545866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":446235284,"num_objects":108948,"num_object_clones":0,"num_object_copies":217896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34988,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":108948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172751,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7150,"num_bytes_recovered":29261843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010388,"kb_used_data":1028244,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281105132,"statfs":{"total":289910292480,"available":287851655168,"internally_reserved":0,"allocated":1052921856,"data_stored":1052050518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1005636609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,3,6,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17874944,"num_objects":-4364,"num_object_clones":0,"num_object_copies":-8728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-443,"num_objects_misplaced":169,"num_objects_unfound":0,"num_objects_dirty":-4364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4364,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003642"},"pg_stats":[{"pgid":"2.f","version":"47'13335","reported_seq":13417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.583540+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:26.583540+0000","last_peered":"2024-09-11T05:19:26.583540+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:26.583540+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28078080,"num_objects":6855,"num_object_clones":0,"num_object_copies":13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3180},{"shards":"0,2","objects":434}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13086","reported_seq":13193,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.831002+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:26.831002+0000","last_peered":"2024-09-11T05:19:26.831002+0000","last_clean":"2024-09-11T05:19:26.831002+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:26.831002+0000","last_undegraded":"2024-09-11T05:19:26.831002+0000","last_fullsized":"2024-09-11T05:19:26.831002+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27910144,"num_objects":6814,"num_object_clones":0,"num_object_copies":13628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11484,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12966","reported_seq":13227,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.889643+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:26.889643+0000","last_peered":"2024-09-11T05:19:26.889643+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:26.889643+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27652120,"num_objects":6752,"num_object_clones":0,"num_object_copies":13504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3054,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":6752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11369,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":375},{"shards":"2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13193","reported_seq":13269,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.846407+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:26.846407+0000","last_peered":"2024-09-11T05:19:26.846407+0000","last_clean":"2024-09-11T05:19:26.846407+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:26.846407+0000","last_undegraded":"2024-09-11T05:19:26.846407+0000","last_fullsized":"2024-09-11T05:19:26.846407+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13193,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13283","reported_seq":13417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.512306+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:26.512306+0000","last_peered":"2024-09-11T05:19:26.512306+0000","last_clean":"2024-09-11T05:19:26.512306+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:26.512306+0000","last_undegraded":"2024-09-11T05:19:26.512306+0000","last_fullsized":"2024-09-11T05:19:26.512306+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10117,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13254","reported_seq":20605,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:28.872505+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:28.872505+0000","last_peered":"2024-09-11T05:19:28.872505+0000","last_clean":"2024-09-11T05:19:28.872505+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:28.872505+0000","last_undegraded":"2024-09-11T05:19:28.872505+0000","last_fullsized":"2024-09-11T05:19:28.872505+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13259","reported_seq":13396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.550572+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:26.550572+0000","last_peered":"2024-09-11T05:19:26.550572+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:26.550572+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28438528,"num_objects":6943,"num_object_clones":0,"num_object_copies":13886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10424,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3367},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13033","reported_seq":13162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.475003+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:26.475003+0000","last_peered":"2024-09-11T05:19:26.475003+0000","last_clean":"2024-09-11T05:19:26.475003+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:26.475003+0000","last_undegraded":"2024-09-11T05:19:26.475003+0000","last_fullsized":"2024-09-11T05:19:26.475003+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28061696,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13132","reported_seq":13268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.870663+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:26.870663+0000","last_peered":"2024-09-11T05:19:26.870663+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:26.870663+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":382},{"shards":"2","objects":3314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704881+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704881+0000","last_peered":"2024-09-11T05:19:25.704881+0000","last_clean":"2024-09-11T05:19:25.704881+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704881+0000","last_undegraded":"2024-09-11T05:19:25.704881+0000","last_fullsized":"2024-09-11T05:19:25.704881+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12903","reported_seq":13073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.583583+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:26.583583+0000","last_peered":"2024-09-11T05:19:26.583583+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:26.583583+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9656,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3327},{"shards":"0,2","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13144","reported_seq":13264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.837021+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:26.837021+0000","last_peered":"2024-09-11T05:19:26.837021+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:26.837021+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28217344,"num_objects":6889,"num_object_clones":0,"num_object_copies":13778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11173,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":397},{"shards":"2","objects":3367}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12940","reported_seq":20030,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.839723+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:26.839723+0000","last_peered":"2024-09-11T05:19:26.839723+0000","last_clean":"2024-09-11T05:19:26.839723+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:26.839723+0000","last_undegraded":"2024-09-11T05:19:26.839723+0000","last_fullsized":"2024-09-11T05:19:26.839723+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":6768,"num_object_clones":0,"num_object_copies":13536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10843,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13276","reported_seq":13407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.547483+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:26.547483+0000","last_peered":"2024-09-11T05:19:26.547483+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:26.547483+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3245},{"shards":"0,2","objects":420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12909","reported_seq":13048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.594159+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:26.594159+0000","last_peered":"2024-09-11T05:19:26.594159+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:26.594159+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27627520,"num_objects":6745,"num_object_clones":0,"num_object_copies":13490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9775,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3250},{"shards":"0,2","objects":417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.706987+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.706987+0000","last_peered":"2024-09-11T05:19:25.706987+0000","last_clean":"2024-09-11T05:19:25.706987+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.706987+0000","last_undegraded":"2024-09-11T05:19:25.706987+0000","last_fullsized":"2024-09-11T05:19:25.706987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13040","reported_seq":13180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.490735+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:26.490735+0000","last_peered":"2024-09-11T05:19:26.490735+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:26.490735+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27688960,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3216},{"shards":"0,2","objects":405}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12932","reported_seq":13080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.531906+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:26.531906+0000","last_peered":"2024-09-11T05:19:26.531906+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:26.531906+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27271785,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9939,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":446235265,"num_objects":108946,"num_object_clones":0,"num_object_copies":217892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34988,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":108946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172749,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7149,"num_bytes_recovered":29261824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050132480,"data_stored":1050054756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621232,"kb_used_data":332700,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93750608,"statfs":{"total":96636764160,"available":96000622592,"internally_reserved":0,"allocated":340684800,"data_stored":340415429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621596,"kb_used_data":310856,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750244,"statfs":{"total":96636764160,"available":96000249856,"internally_reserved":0,"allocated":318316544,"data_stored":317983953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215187,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767560,"kb_used_data":384688,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93604280,"statfs":{"total":96636764160,"available":95850782720,"internally_reserved":0,"allocated":393920512,"data_stored":393651136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":392384512,"data_stored":392381033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317706240,"data_stored":317639546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340041728,"data_stored":340034177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:30.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:30.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13335", 'reported_seq': 13417, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.583540+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:26.583540+0000', 'last_peered': '2024-09-11T05:19:26.583540+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:26.583540+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28078080, 'num_objects': 6855, 'num_object_clones': 0, 'num_object_copies': 13710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6855, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13335, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3180}, {'shards': '0,2', 'objects': 434}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:30.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:30.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'12966", 'reported_seq': 13227, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.889643+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:19:26.889643+0000', 'last_peered': '2024-09-11T05:19:26.889643+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:26.889643+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27652120, 'num_objects': 6752, 'num_object_clones': 0, 'num_object_copies': 13504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3054, 'num_objects_misplaced': 375, 'num_objects_unfound': 0, 'num_objects_dirty': 6752, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11369, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 266240, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 375}, {'shards': '2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:30.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:30.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13259", 'reported_seq': 13396, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.550572+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:26.550572+0000', 'last_peered': '2024-09-11T05:19:26.550572+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:26.550572+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28438528, 'num_objects': 6943, 'num_object_clones': 0, 'num_object_copies': 13886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6943, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10424, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3367}, {'shards': '0,2', 'objects': 421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:30.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:30.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13132", 'reported_seq': 13268, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.870663+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:26.870663+0000', 'last_peered': '2024-09-11T05:19:26.870663+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:26.870663+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28012544, 'num_objects': 6839, 'num_object_clones': 0, 'num_object_copies': 13678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11200, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 382}, {'shards': '2', 'objects': 3314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:30.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12903", 'reported_seq': 13073, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.583583+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:26.583583+0000', 'last_peered': '2024-09-11T05:19:26.583583+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:26.583583+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27725824, 'num_objects': 6769, 'num_object_clones': 0, 'num_object_copies': 13538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9656, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3327}, {'shards': '0,2', 'objects': 379}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13144", 'reported_seq': 13264, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.837021+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:26.837021+0000', 'last_peered': '2024-09-11T05:19:26.837021+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:26.837021+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "39'3100", 'ondisk_log_start': "39'3100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28217344, 'num_objects': 6889, 'num_object_clones': 0, 'num_object_copies': 13778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11173, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 397}, {'shards': '2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13276", 'reported_seq': 13407, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.547483+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:26.547483+0000', 'last_peered': '2024-09-11T05:19:26.547483+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:26.547483+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28131328, 'num_objects': 6868, 'num_object_clones': 0, 'num_object_copies': 13736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10309, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3245}, {'shards': '0,2', 'objects': 420}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12909", 'reported_seq': 13048, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.594159+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:26.594159+0000', 'last_peered': '2024-09-11T05:19:26.594159+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:26.594159+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27627520, 'num_objects': 6745, 'num_object_clones': 0, 'num_object_copies': 13490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9775, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3250}, {'shards': '0,2', 'objects': 417}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13040", 'reported_seq': 13180, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.490735+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:26.490735+0000', 'last_peered': '2024-09-11T05:19:26.490735+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:26.490735+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27688960, 'num_objects': 6760, 'num_object_clones': 0, 'num_object_copies': 13520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6760, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9798, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3216}, {'shards': '0,2', 'objects': 405}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:30.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:30.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'12932", 'reported_seq': 13080, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:26.531906+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:26.531906+0000', 'last_peered': '2024-09-11T05:19:26.531906+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:26.531906+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27271785, 'num_objects': 6659, 'num_object_clones': 0, 'num_object_copies': 13318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9939, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3144}, {'shards': '0,2', 'objects': 384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:30.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:31.714 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:31.715 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34988/217896 objects degraded (16.057%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":108948,"data_bytes":446235284,"bytes_used":2058637312,"bytes_avail":287851655168,"bytes_total":289910292480,"degraded_objects":34988,"degraded_total":217896,"degraded_ratio":0.16057201600763668,"misplaced_objects":375,"misplaced_total":217896,"misplaced_ratio":0.0017210045159158497,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":67552,"recovering_keys_per_sec":0,"num_objects_recovered":133,"num_bytes_recovered":540672,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":270},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:31.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:31.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:32.580 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:32.580 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:32.651 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-11T05:19:31.546401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":446235284,"num_objects":108948,"num_object_clones":0,"num_object_copies":217896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34988,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":108948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172751,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7150,"num_bytes_recovered":29261843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010388,"kb_used_data":1028244,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281105132,"statfs":{"total":289910292480,"available":287851655168,"internally_reserved":0,"allocated":1052921856,"data_stored":1052050518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1005636609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,3,6,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8855552,"num_objects":-2162,"num_object_clones":0,"num_object_copies":-4324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-443,"num_objects_misplaced":310,"num_objects_unfound":0,"num_objects_dirty":-2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":540672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003738"},"pg_stats":[{"pgid":"2.f","version":"47'13335","reported_seq":13417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.583540+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:26.583540+0000","last_peered":"2024-09-11T05:19:26.583540+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:26.583540+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28078080,"num_objects":6855,"num_object_clones":0,"num_object_copies":13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3180},{"shards":"0,2","objects":434}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13086","reported_seq":13193,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.831002+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:26.831002+0000","last_peered":"2024-09-11T05:19:26.831002+0000","last_clean":"2024-09-11T05:19:26.831002+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:26.831002+0000","last_undegraded":"2024-09-11T05:19:26.831002+0000","last_fullsized":"2024-09-11T05:19:26.831002+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27910144,"num_objects":6814,"num_object_clones":0,"num_object_copies":13628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11484,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12966","reported_seq":13227,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.889643+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:26.889643+0000","last_peered":"2024-09-11T05:19:26.889643+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:26.889643+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27652120,"num_objects":6752,"num_object_clones":0,"num_object_copies":13504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3054,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":6752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11369,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":266240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":375},{"shards":"2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13193","reported_seq":13269,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.846407+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:26.846407+0000","last_peered":"2024-09-11T05:19:26.846407+0000","last_clean":"2024-09-11T05:19:26.846407+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:26.846407+0000","last_undegraded":"2024-09-11T05:19:26.846407+0000","last_fullsized":"2024-09-11T05:19:26.846407+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13193,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13283","reported_seq":13417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.512306+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:26.512306+0000","last_peered":"2024-09-11T05:19:26.512306+0000","last_clean":"2024-09-11T05:19:26.512306+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:26.512306+0000","last_undegraded":"2024-09-11T05:19:26.512306+0000","last_fullsized":"2024-09-11T05:19:26.512306+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10117,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13254","reported_seq":20605,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:28.872505+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:28.872505+0000","last_peered":"2024-09-11T05:19:28.872505+0000","last_clean":"2024-09-11T05:19:28.872505+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:28.872505+0000","last_undegraded":"2024-09-11T05:19:28.872505+0000","last_fullsized":"2024-09-11T05:19:28.872505+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13259","reported_seq":13396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.550572+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:26.550572+0000","last_peered":"2024-09-11T05:19:26.550572+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:26.550572+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28438528,"num_objects":6943,"num_object_clones":0,"num_object_copies":13886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10424,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3367},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13033","reported_seq":13162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.475003+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:26.475003+0000","last_peered":"2024-09-11T05:19:26.475003+0000","last_clean":"2024-09-11T05:19:26.475003+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:26.475003+0000","last_undegraded":"2024-09-11T05:19:26.475003+0000","last_fullsized":"2024-09-11T05:19:26.475003+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28061696,"num_objects":6851,"num_object_clones":0,"num_object_copies":13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13132","reported_seq":13268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.870663+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:26.870663+0000","last_peered":"2024-09-11T05:19:26.870663+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:26.870663+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":382},{"shards":"2","objects":3314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704881+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704881+0000","last_peered":"2024-09-11T05:19:25.704881+0000","last_clean":"2024-09-11T05:19:25.704881+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704881+0000","last_undegraded":"2024-09-11T05:19:25.704881+0000","last_fullsized":"2024-09-11T05:19:25.704881+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12903","reported_seq":13073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.583583+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:26.583583+0000","last_peered":"2024-09-11T05:19:26.583583+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:26.583583+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27725824,"num_objects":6769,"num_object_clones":0,"num_object_copies":13538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9656,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3327},{"shards":"0,2","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13144","reported_seq":13264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.837021+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:26.837021+0000","last_peered":"2024-09-11T05:19:26.837021+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:26.837021+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28217344,"num_objects":6889,"num_object_clones":0,"num_object_copies":13778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11173,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":397},{"shards":"2","objects":3367}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12940","reported_seq":20030,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:26.839723+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:26.839723+0000","last_peered":"2024-09-11T05:19:26.839723+0000","last_clean":"2024-09-11T05:19:26.839723+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:26.839723+0000","last_undegraded":"2024-09-11T05:19:26.839723+0000","last_fullsized":"2024-09-11T05:19:26.839723+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":6768,"num_object_clones":0,"num_object_copies":13536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10843,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13276","reported_seq":13407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.547483+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:26.547483+0000","last_peered":"2024-09-11T05:19:26.547483+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:26.547483+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3245},{"shards":"0,2","objects":420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12909","reported_seq":13048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.594159+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:26.594159+0000","last_peered":"2024-09-11T05:19:26.594159+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:26.594159+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27627520,"num_objects":6745,"num_object_clones":0,"num_object_copies":13490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9775,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3250},{"shards":"0,2","objects":417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.706987+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.706987+0000","last_peered":"2024-09-11T05:19:25.706987+0000","last_clean":"2024-09-11T05:19:25.706987+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.706987+0000","last_undegraded":"2024-09-11T05:19:25.706987+0000","last_fullsized":"2024-09-11T05:19:25.706987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13040","reported_seq":13180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.490735+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:26.490735+0000","last_peered":"2024-09-11T05:19:26.490735+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:26.490735+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27688960,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3216},{"shards":"0,2","objects":405}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12932","reported_seq":13080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:26.531906+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:26.531906+0000","last_peered":"2024-09-11T05:19:26.531906+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:26.531906+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27271785,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9939,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":446235265,"num_objects":108946,"num_object_clones":0,"num_object_copies":217892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34988,"num_objects_misplaced":375,"num_objects_unfound":0,"num_objects_dirty":108946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172749,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7149,"num_bytes_recovered":29261824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050132480,"data_stored":1050054756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621232,"kb_used_data":332700,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93750608,"statfs":{"total":96636764160,"available":96000622592,"internally_reserved":0,"allocated":340684800,"data_stored":340415429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621596,"kb_used_data":310856,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750244,"statfs":{"total":96636764160,"available":96000249856,"internally_reserved":0,"allocated":318316544,"data_stored":317983953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215187,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767560,"kb_used_data":384688,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93604280,"statfs":{"total":96636764160,"available":95850782720,"internally_reserved":0,"allocated":393920512,"data_stored":393651136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":392384512,"data_stored":392381033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317706240,"data_stored":317639546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340041728,"data_stored":340034177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:35.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:36.542 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:36.543 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:36.589 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-11T05:19:35.552196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437129876,"num_objects":106725,"num_object_clones":0,"num_object_copies":213450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35003,"num_objects_misplaced":267,"num_objects_unfound":0,"num_objects_dirty":106725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174974,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7243,"num_bytes_recovered":29642771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1985400,"kb_used_data":1015744,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281130120,"statfs":{"total":289910292480,"available":287877242880,"internally_reserved":0,"allocated":1040121856,"data_stored":1039250518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":992857089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17960960,"num_objects":-4385,"num_object_clones":0,"num_object_copies":-8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-428,"num_objects_misplaced":202,"num_objects_unfound":0,"num_objects_dirty":-4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":921600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009025"},"pg_stats":[{"pgid":"2.f","version":"47'13488","reported_seq":13571,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.613318+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:31.613318+0000","last_peered":"2024-09-11T05:19:31.613318+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:31.613318+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13488,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3027},{"shards":"0,2","objects":281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13216","reported_seq":13323,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.879759+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:31.879759+0000","last_peered":"2024-09-11T05:19:31.879759+0000","last_clean":"2024-09-11T05:19:31.879759+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:31.879759+0000","last_undegraded":"2024-09-11T05:19:31.879759+0000","last_fullsized":"2024-09-11T05:19:31.879759+0000","mapping_epoch":35,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27377664,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11614,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13088","reported_seq":13537,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.855317+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:31.855317+0000","last_peered":"2024-09-11T05:19:31.855317+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:31.855317+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152408,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3069,"num_objects_misplaced":267,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11491,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":647168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":267},{"shards":"2","objects":3294}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13359","reported_seq":13435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.876616+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:31.876616+0000","last_peered":"2024-09-11T05:19:31.876616+0000","last_clean":"2024-09-11T05:19:31.876616+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:31.876616+0000","last_undegraded":"2024-09-11T05:19:31.876616+0000","last_fullsized":"2024-09-11T05:19:31.876616+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13359,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13448","reported_seq":13578,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.616429+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:31.616429+0000","last_peered":"2024-09-11T05:19:31.616429+0000","last_clean":"2024-09-11T05:19:31.616429+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:31.616429+0000","last_undegraded":"2024-09-11T05:19:31.616429+0000","last_fullsized":"2024-09-11T05:19:31.616429+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529216,"num_objects":6721,"num_object_clones":0,"num_object_copies":13442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10282,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13396","reported_seq":20747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:33.909201+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:33.909201+0000","last_peered":"2024-09-11T05:19:33.909201+0000","last_clean":"2024-09-11T05:19:33.909201+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:33.909201+0000","last_undegraded":"2024-09-11T05:19:33.909201+0000","last_fullsized":"2024-09-11T05:19:33.909201+0000","mapping_epoch":46,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9940,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13401","reported_seq":13537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.615937+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:31.615937+0000","last_peered":"2024-09-11T05:19:31.615937+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:31.615937+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27856896,"num_objects":6801,"num_object_clones":0,"num_object_copies":13602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13186","reported_seq":13315,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.579879+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:31.579879+0000","last_peered":"2024-09-11T05:19:31.579879+0000","last_clean":"2024-09-11T05:19:31.579879+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:31.579879+0000","last_undegraded":"2024-09-11T05:19:31.579879+0000","last_fullsized":"2024-09-11T05:19:31.579879+0000","mapping_epoch":34,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27435008,"num_objects":6698,"num_object_clones":0,"num_object_copies":13396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13253","reported_seq":13389,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.796587+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:31.796587+0000","last_peered":"2024-09-11T05:19:31.796587+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:31.796587+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11321,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":261},{"shards":"2","objects":3193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13037","reported_seq":13207,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.601284+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:31.601284+0000","last_peered":"2024-09-11T05:19:31.601284+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:31.601284+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9790,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3193},{"shards":"0,2","objects":245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13261","reported_seq":13384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.787189+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:31.787189+0000","last_peered":"2024-09-11T05:19:31.787189+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:31.787189+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11290,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":280},{"shards":"2","objects":3250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13077","reported_seq":20167,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.899758+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:31.899758+0000","last_peered":"2024-09-11T05:19:31.899758+0000","last_clean":"2024-09-11T05:19:31.899758+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:31.899758+0000","last_undegraded":"2024-09-11T05:19:31.899758+0000","last_fullsized":"2024-09-11T05:19:31.899758+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27160576,"num_objects":6631,"num_object_clones":0,"num_object_copies":13262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10980,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13420","reported_seq":13556,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.615327+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:31.615327+0000","last_peered":"2024-09-11T05:19:31.615327+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:31.615327+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10453,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3101},{"shards":"0,2","objects":276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13053","reported_seq":13192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.673088+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:31.673088+0000","last_peered":"2024-09-11T05:19:31.673088+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:31.673088+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9919,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3106},{"shards":"0,2","objects":273}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13162","reported_seq":13302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.728581+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:31.728581+0000","last_peered":"2024-09-11T05:19:31.728581+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:31.728581+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27189248,"num_objects":6638,"num_object_clones":0,"num_object_copies":13276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9920,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3094},{"shards":"0,2","objects":283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13063","reported_seq":13211,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.616968+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:31.616968+0000","last_peered":"2024-09-11T05:19:31.616968+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:31.616968+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735209,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10070,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3013},{"shards":"0,2","objects":253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437129857,"num_objects":106723,"num_object_clones":0,"num_object_copies":213446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35003,"num_objects_misplaced":267,"num_objects_unfound":0,"num_objects_dirty":106723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174972,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7242,"num_bytes_recovered":29642752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1037303808,"data_stored":1037226084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620968,"kb_used_data":328272,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93750872,"statfs":{"total":96636764160,"available":96000892928,"internally_reserved":0,"allocated":336150528,"data_stored":335881157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612100,"kb_used_data":309684,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93759740,"statfs":{"total":96636764160,"available":96009973760,"internally_reserved":0,"allocated":317116416,"data_stored":316783825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215188,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752332,"kb_used_data":377788,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93619508,"statfs":{"total":96636764160,"available":95866376192,"internally_reserved":0,"allocated":386854912,"data_stored":386585536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385273856,"data_stored":385270377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316530688,"data_stored":316463994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335499264,"data_stored":335491713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:36.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:37.385 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:37.388 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:37.429 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-11T05:19:35.552196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437129876,"num_objects":106725,"num_object_clones":0,"num_object_copies":213450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35003,"num_objects_misplaced":267,"num_objects_unfound":0,"num_objects_dirty":106725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174974,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7243,"num_bytes_recovered":29642771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1985400,"kb_used_data":1015744,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281130120,"statfs":{"total":289910292480,"available":287877242880,"internally_reserved":0,"allocated":1040121856,"data_stored":1039250518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":992857089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17960960,"num_objects":-4385,"num_object_clones":0,"num_object_copies":-8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-428,"num_objects_misplaced":202,"num_objects_unfound":0,"num_objects_dirty":-4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":921600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009025"},"pg_stats":[{"pgid":"2.f","version":"47'13488","reported_seq":13571,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.613318+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:31.613318+0000","last_peered":"2024-09-11T05:19:31.613318+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:31.613318+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13488,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3027},{"shards":"0,2","objects":281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13216","reported_seq":13323,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.879759+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:31.879759+0000","last_peered":"2024-09-11T05:19:31.879759+0000","last_clean":"2024-09-11T05:19:31.879759+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:31.879759+0000","last_undegraded":"2024-09-11T05:19:31.879759+0000","last_fullsized":"2024-09-11T05:19:31.879759+0000","mapping_epoch":35,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27377664,"num_objects":6684,"num_object_clones":0,"num_object_copies":13368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11614,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13088","reported_seq":13537,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.855317+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:31.855317+0000","last_peered":"2024-09-11T05:19:31.855317+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:31.855317+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152408,"num_objects":6630,"num_object_clones":0,"num_object_copies":13260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3069,"num_objects_misplaced":267,"num_objects_unfound":0,"num_objects_dirty":6630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11491,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":647168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":267},{"shards":"2","objects":3294}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13359","reported_seq":13435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.876616+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:31.876616+0000","last_peered":"2024-09-11T05:19:31.876616+0000","last_clean":"2024-09-11T05:19:31.876616+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:31.876616+0000","last_undegraded":"2024-09-11T05:19:31.876616+0000","last_fullsized":"2024-09-11T05:19:31.876616+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13359,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13448","reported_seq":13578,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.616429+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:31.616429+0000","last_peered":"2024-09-11T05:19:31.616429+0000","last_clean":"2024-09-11T05:19:31.616429+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:31.616429+0000","last_undegraded":"2024-09-11T05:19:31.616429+0000","last_fullsized":"2024-09-11T05:19:31.616429+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529216,"num_objects":6721,"num_object_clones":0,"num_object_copies":13442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10282,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13396","reported_seq":20747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:33.909201+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:33.909201+0000","last_peered":"2024-09-11T05:19:33.909201+0000","last_clean":"2024-09-11T05:19:33.909201+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:33.909201+0000","last_undegraded":"2024-09-11T05:19:33.909201+0000","last_fullsized":"2024-09-11T05:19:33.909201+0000","mapping_epoch":46,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9940,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13401","reported_seq":13537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.615937+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:31.615937+0000","last_peered":"2024-09-11T05:19:31.615937+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:31.615937+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27856896,"num_objects":6801,"num_object_clones":0,"num_object_copies":13602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13186","reported_seq":13315,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.579879+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:31.579879+0000","last_peered":"2024-09-11T05:19:31.579879+0000","last_clean":"2024-09-11T05:19:31.579879+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:31.579879+0000","last_undegraded":"2024-09-11T05:19:31.579879+0000","last_fullsized":"2024-09-11T05:19:31.579879+0000","mapping_epoch":34,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27435008,"num_objects":6698,"num_object_clones":0,"num_object_copies":13396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13253","reported_seq":13389,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.796587+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:31.796587+0000","last_peered":"2024-09-11T05:19:31.796587+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:31.796587+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11321,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":261},{"shards":"2","objects":3193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13037","reported_seq":13207,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.601284+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:31.601284+0000","last_peered":"2024-09-11T05:19:31.601284+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:31.601284+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9790,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3193},{"shards":"0,2","objects":245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13261","reported_seq":13384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.787189+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:31.787189+0000","last_peered":"2024-09-11T05:19:31.787189+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:31.787189+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11290,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":280},{"shards":"2","objects":3250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13077","reported_seq":20167,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.899758+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:31.899758+0000","last_peered":"2024-09-11T05:19:31.899758+0000","last_clean":"2024-09-11T05:19:31.899758+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:31.899758+0000","last_undegraded":"2024-09-11T05:19:31.899758+0000","last_fullsized":"2024-09-11T05:19:31.899758+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27160576,"num_objects":6631,"num_object_clones":0,"num_object_copies":13262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10980,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13420","reported_seq":13556,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.615327+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:31.615327+0000","last_peered":"2024-09-11T05:19:31.615327+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:31.615327+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10453,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3101},{"shards":"0,2","objects":276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13053","reported_seq":13192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.673088+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:31.673088+0000","last_peered":"2024-09-11T05:19:31.673088+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:31.673088+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9919,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3106},{"shards":"0,2","objects":273}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13162","reported_seq":13302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.728581+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:31.728581+0000","last_peered":"2024-09-11T05:19:31.728581+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:31.728581+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27189248,"num_objects":6638,"num_object_clones":0,"num_object_copies":13276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9920,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3094},{"shards":"0,2","objects":283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13063","reported_seq":13211,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:31.616968+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:31.616968+0000","last_peered":"2024-09-11T05:19:31.616968+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:31.616968+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735209,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10070,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3013},{"shards":"0,2","objects":253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437129857,"num_objects":106723,"num_object_clones":0,"num_object_copies":213446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35003,"num_objects_misplaced":267,"num_objects_unfound":0,"num_objects_dirty":106723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174972,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7242,"num_bytes_recovered":29642752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1037303808,"data_stored":1037226084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620968,"kb_used_data":328272,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93750872,"statfs":{"total":96636764160,"available":96000892928,"internally_reserved":0,"allocated":336150528,"data_stored":335881157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612100,"kb_used_data":309684,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93759740,"statfs":{"total":96636764160,"available":96009973760,"internally_reserved":0,"allocated":317116416,"data_stored":316783825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215188,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752332,"kb_used_data":377788,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93619508,"statfs":{"total":96636764160,"available":95866376192,"internally_reserved":0,"allocated":386854912,"data_stored":386585536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385273856,"data_stored":385270377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316530688,"data_stored":316463994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335499264,"data_stored":335491713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13488", 'reported_seq': 13571, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.613318+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:31.613318+0000', 'last_peered': '2024-09-11T05:19:31.613318+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:31.613318+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27451392, 'num_objects': 6702, 'num_object_clones': 0, 'num_object_copies': 13404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13488, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3027}, {'shards': '0,2', 'objects': 281}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13088", 'reported_seq': 13537, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.855317+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:19:31.855317+0000', 'last_peered': '2024-09-11T05:19:31.855317+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:31.855317+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27152408, 'num_objects': 6630, 'num_object_clones': 0, 'num_object_copies': 13260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3069, 'num_objects_misplaced': 267, 'num_objects_unfound': 0, 'num_objects_dirty': 6630, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11491, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 159, 'num_bytes_recovered': 647168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 267}, {'shards': '2', 'objects': 3294}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13401", 'reported_seq': 13537, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.615937+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:31.615937+0000', 'last_peered': '2024-09-11T05:19:31.615937+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:31.615937+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27856896, 'num_objects': 6801, 'num_object_clones': 0, 'num_object_copies': 13602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6801, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10566, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3225}, {'shards': '0,2', 'objects': 279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13253", 'reported_seq': 13389, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.796587+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:31.796587+0000', 'last_peered': '2024-09-11T05:19:31.796587+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:31.796587+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27516928, 'num_objects': 6718, 'num_object_clones': 0, 'num_object_copies': 13436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11321, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 261}, {'shards': '2', 'objects': 3193}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13037", 'reported_seq': 13207, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.601284+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:31.601284+0000', 'last_peered': '2024-09-11T05:19:31.601284+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:31.601284+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27176960, 'num_objects': 6635, 'num_object_clones': 0, 'num_object_copies': 13270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9790, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3193}, {'shards': '0,2', 'objects': 245}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13261", 'reported_seq': 13384, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.787189+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:31.787189+0000', 'last_peered': '2024-09-11T05:19:31.787189+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:31.787189+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27738112, 'num_objects': 6772, 'num_object_clones': 0, 'num_object_copies': 13544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11290, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 280}, {'shards': '2', 'objects': 3250}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13420", 'reported_seq': 13556, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.615327+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:31.615327+0000', 'last_peered': '2024-09-11T05:19:31.615327+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:31.615327+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27541504, 'num_objects': 6724, 'num_object_clones': 0, 'num_object_copies': 13448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6724, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10453, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3101}, {'shards': '0,2', 'objects': 276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13053", 'reported_seq': 13192, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.673088+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:31.673088+0000', 'last_peered': '2024-09-11T05:19:31.673088+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:31.673088+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27037696, 'num_objects': 6601, 'num_object_clones': 0, 'num_object_copies': 13202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9919, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3106}, {'shards': '0,2', 'objects': 273}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13162", 'reported_seq': 13302, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.728581+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:31.728581+0000', 'last_peered': '2024-09-11T05:19:31.728581+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:31.728581+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27189248, 'num_objects': 6638, 'num_object_clones': 0, 'num_object_copies': 13276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9920, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3094}, {'shards': '0,2', 'objects': 283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13063", 'reported_seq': 13211, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:31.616968+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:31.616968+0000', 'last_peered': '2024-09-11T05:19:31.616968+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:31.616968+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26735209, 'num_objects': 6528, 'num_object_clones': 0, 'num_object_copies': 13056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10070, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3013}, {'shards': '0,2', 'objects': 253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:37.432 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:38.465 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:38.465 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35071/209258 objects degraded (16.760%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":456,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":104629,"data_bytes":428544660,"bytes_used":2033258496,"bytes_avail":287877033984,"bytes_total":289910292480,"degraded_objects":35071,"degraded_total":209258,"degraded_ratio":0.16759693775148382,"misplaced_objects":114,"misplaced_total":209258,"misplaced_ratio":0.00054478203939634323,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":107158,"recovering_keys_per_sec":0,"num_objects_recovered":311,"num_bytes_recovered":1269760,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":546},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:38.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:38.586 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:39.364 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:39.364 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:39.415 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-11T05:19:37.552657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":428544660,"num_objects":104629,"num_object_clones":0,"num_object_copies":209258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35071,"num_objects_misplaced":114,"num_objects_unfound":0,"num_objects_dirty":104629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177070,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7328,"num_bytes_recovered":29990931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1985604,"kb_used_data":1003456,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281129916,"statfs":{"total":289910292480,"available":287877033984,"internally_reserved":0,"allocated":1027538944,"data_stored":1026667606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1005636609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,5,6,9,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26546176,"num_objects":-6481,"num_object_clones":0,"num_object_copies":-12962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-360,"num_objects_misplaced":49,"num_objects_unfound":0,"num_objects_dirty":-6481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6481,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.849316"},"pg_stats":[{"pgid":"2.f","version":"47'13632","reported_seq":13715,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.712576+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:36.712576+0000","last_peered":"2024-09-11T05:19:36.712576+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:36.712576+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861568,"num_objects":6558,"num_object_clones":0,"num_object_copies":13116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13632,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2883},{"shards":"0,2","objects":137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13374","reported_seq":13481,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.874940+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:36.874940+0000","last_peered":"2024-09-11T05:19:36.874940+0000","last_clean":"2024-09-11T05:19:36.874940+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:36.874940+0000","last_undegraded":"2024-09-11T05:19:36.874940+0000","last_fullsized":"2024-09-11T05:19:36.874940+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26730496,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11772,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13239","reported_seq":13859,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.865880+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:36.865880+0000","last_peered":"2024-09-11T05:19:36.865880+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:36.865880+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26533912,"num_objects":6479,"num_object_clones":0,"num_object_copies":12958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3137,"num_objects_misplaced":114,"num_objects_unfound":0,"num_objects_dirty":6479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11642,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":114},{"shards":"2","objects":3228}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13493","reported_seq":13569,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.840912+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:36.840912+0000","last_peered":"2024-09-11T05:19:36.840912+0000","last_clean":"2024-09-11T05:19:36.840912+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:36.840912+0000","last_undegraded":"2024-09-11T05:19:36.840912+0000","last_fullsized":"2024-09-11T05:19:36.840912+0000","mapping_epoch":35,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13571","reported_seq":13706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.715946+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:36.715946+0000","last_peered":"2024-09-11T05:19:36.715946+0000","last_clean":"2024-09-11T05:19:36.715946+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:36.715946+0000","last_undegraded":"2024-09-11T05:19:36.715946+0000","last_fullsized":"2024-09-11T05:19:36.715946+0000","mapping_epoch":34,"log_start":"39'3500","ondisk_log_start":"39'3500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10405,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13396","reported_seq":20747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:33.909201+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:33.909201+0000","last_peered":"2024-09-11T05:19:33.909201+0000","last_clean":"2024-09-11T05:19:33.909201+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:33.909201+0000","last_undegraded":"2024-09-11T05:19:33.909201+0000","last_fullsized":"2024-09-11T05:19:33.909201+0000","mapping_epoch":46,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9940,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13545","reported_seq":13680,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.775172+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:36.775172+0000","last_peered":"2024-09-11T05:19:36.775172+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:36.775172+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10710,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13323","reported_seq":13452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.730090+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:36.730090+0000","last_peered":"2024-09-11T05:19:36.730090+0000","last_clean":"2024-09-11T05:19:36.730090+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:36.730090+0000","last_undegraded":"2024-09-11T05:19:36.730090+0000","last_fullsized":"2024-09-11T05:19:36.730090+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10626,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13391","reported_seq":13527,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.789597+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:36.789597+0000","last_peered":"2024-09-11T05:19:36.789597+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:36.789597+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26951680,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11459,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":123},{"shards":"2","objects":3055}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13162","reported_seq":13332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.700383+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:36.700383+0000","last_peered":"2024-09-11T05:19:36.700383+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:36.700383+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26664960,"num_objects":6510,"num_object_clones":0,"num_object_copies":13020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3068},{"shards":"0,2","objects":120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13411","reported_seq":13534,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.800789+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:36.800789+0000","last_peered":"2024-09-11T05:19:36.800789+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:36.800789+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27123712,"num_objects":6622,"num_object_clones":0,"num_object_copies":13244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11440,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":130},{"shards":"2","objects":3100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13211","reported_seq":20301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.885518+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:36.885518+0000","last_peered":"2024-09-11T05:19:36.885518+0000","last_clean":"2024-09-11T05:19:36.885518+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:36.885518+0000","last_undegraded":"2024-09-11T05:19:36.885518+0000","last_fullsized":"2024-09-11T05:19:36.885518+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26611712,"num_objects":6497,"num_object_clones":0,"num_object_copies":12994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11114,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13557","reported_seq":13692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.755812+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:36.755812+0000","last_peered":"2024-09-11T05:19:36.755812+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:36.755812+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10590,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2964},{"shards":"0,2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13180","reported_seq":13319,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.628508+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:36.628508+0000","last_peered":"2024-09-11T05:19:36.628508+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:36.628508+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10046,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2979},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13306","reported_seq":13446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.773918+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:36.773918+0000","last_peered":"2024-09-11T05:19:36.773918+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:36.773918+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10064,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2950},{"shards":"0,2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13213","reported_seq":13362,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.741507+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:36.741507+0000","last_peered":"2024-09-11T05:19:36.741507+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:36.741507+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120809,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10220,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2863},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":428544641,"num_objects":104627,"num_object_clones":0,"num_object_copies":209254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35071,"num_objects_misplaced":114,"num_objects_unfound":0,"num_objects_dirty":104627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177068,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7327,"num_bytes_recovered":29990912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1025818624,"data_stored":1025740900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620684,"kb_used_data":323824,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93751156,"statfs":{"total":96636764160,"available":96001183744,"internally_reserved":0,"allocated":331595776,"data_stored":331326405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612100,"kb_used_data":309684,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93759740,"statfs":{"total":96636764160,"available":96009973760,"internally_reserved":0,"allocated":317116416,"data_stored":316783825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215189,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752820,"kb_used_data":369948,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93619020,"statfs":{"total":96636764160,"available":95865876480,"internally_reserved":0,"allocated":378826752,"data_stored":378557376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378392576,"data_stored":378389097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316530688,"data_stored":316463994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330895360,"data_stored":330887809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:42.417 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:43.205 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:43.208 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:43.271 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-11T05:19:41.553708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":427885204,"num_objects":104468,"num_object_clones":0,"num_object_copies":208936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35071,"num_objects_misplaced":114,"num_objects_unfound":0,"num_objects_dirty":104468,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177231,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7328,"num_bytes_recovered":29990931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1988572,"kb_used_data":1002264,"kb_used_omap":12,"kb_used_meta":986227,"kb_avail":281126948,"statfs":{"total":289910292480,"available":287873994752,"internally_reserved":0,"allocated":1026318336,"data_stored":1025446998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1009896449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,4,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18350080,"num_objects":-4480,"num_object_clones":0,"num_object_copies":-8960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":83,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":-4480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4480,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":729088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007841"},"pg_stats":[{"pgid":"2.f","version":"47'13632","reported_seq":13715,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.712576+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:36.712576+0000","last_peered":"2024-09-11T05:19:36.712576+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:36.712576+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"40'3600","ondisk_log_start":"40'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861568,"num_objects":6558,"num_object_clones":0,"num_object_copies":13116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13632,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2883},{"shards":"0,2","objects":137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13374","reported_seq":13481,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.874940+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:36.874940+0000","last_peered":"2024-09-11T05:19:36.874940+0000","last_clean":"2024-09-11T05:19:36.874940+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:36.874940+0000","last_undegraded":"2024-09-11T05:19:36.874940+0000","last_fullsized":"2024-09-11T05:19:36.874940+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26730496,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11772,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13239","reported_seq":13859,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.865880+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:36.865880+0000","last_peered":"2024-09-11T05:19:36.865880+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:36.865880+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26533912,"num_objects":6479,"num_object_clones":0,"num_object_copies":12958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3137,"num_objects_misplaced":114,"num_objects_unfound":0,"num_objects_dirty":6479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11642,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":114},{"shards":"2","objects":3228}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13493","reported_seq":13569,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.840912+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:36.840912+0000","last_peered":"2024-09-11T05:19:36.840912+0000","last_clean":"2024-09-11T05:19:36.840912+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:36.840912+0000","last_undegraded":"2024-09-11T05:19:36.840912+0000","last_fullsized":"2024-09-11T05:19:36.840912+0000","mapping_epoch":35,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13571","reported_seq":13706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.715946+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:36.715946+0000","last_peered":"2024-09-11T05:19:36.715946+0000","last_clean":"2024-09-11T05:19:36.715946+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:36.715946+0000","last_undegraded":"2024-09-11T05:19:36.715946+0000","last_fullsized":"2024-09-11T05:19:36.715946+0000","mapping_epoch":34,"log_start":"39'3500","ondisk_log_start":"39'3500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10405,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13557","reported_seq":20908,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:38.990825+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:38.990825+0000","last_peered":"2024-09-11T05:19:38.990825+0000","last_clean":"2024-09-11T05:19:38.990825+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:38.990825+0000","last_undegraded":"2024-09-11T05:19:38.990825+0000","last_fullsized":"2024-09-11T05:19:38.990825+0000","mapping_epoch":46,"log_start":"39'3500","ondisk_log_start":"39'3500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26533888,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13545","reported_seq":13680,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.775172+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:36.775172+0000","last_peered":"2024-09-11T05:19:36.775172+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:36.775172+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10710,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13323","reported_seq":13452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.730090+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:36.730090+0000","last_peered":"2024-09-11T05:19:36.730090+0000","last_clean":"2024-09-11T05:19:36.730090+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:36.730090+0000","last_undegraded":"2024-09-11T05:19:36.730090+0000","last_fullsized":"2024-09-11T05:19:36.730090+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10626,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13391","reported_seq":13527,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.789597+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:36.789597+0000","last_peered":"2024-09-11T05:19:36.789597+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:36.789597+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26951680,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11459,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":123},{"shards":"2","objects":3055}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13162","reported_seq":13332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.700383+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:36.700383+0000","last_peered":"2024-09-11T05:19:36.700383+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:36.700383+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26664960,"num_objects":6510,"num_object_clones":0,"num_object_copies":13020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3068},{"shards":"0,2","objects":120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13411","reported_seq":13534,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.800789+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:36.800789+0000","last_peered":"2024-09-11T05:19:36.800789+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:36.800789+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27123712,"num_objects":6622,"num_object_clones":0,"num_object_copies":13244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11440,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":130},{"shards":"2","objects":3100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13211","reported_seq":20301,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:36.885518+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:36.885518+0000","last_peered":"2024-09-11T05:19:36.885518+0000","last_clean":"2024-09-11T05:19:36.885518+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:36.885518+0000","last_undegraded":"2024-09-11T05:19:36.885518+0000","last_fullsized":"2024-09-11T05:19:36.885518+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26611712,"num_objects":6497,"num_object_clones":0,"num_object_copies":12994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11114,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13557","reported_seq":13692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.755812+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:36.755812+0000","last_peered":"2024-09-11T05:19:36.755812+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:36.755812+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"39'3500","ondisk_log_start":"39'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10590,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2964},{"shards":"0,2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13180","reported_seq":13319,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.628508+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:36.628508+0000","last_peered":"2024-09-11T05:19:36.628508+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:36.628508+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"39'3100","ondisk_log_start":"39'3100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10046,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2979},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13306","reported_seq":13446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.773918+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:36.773918+0000","last_peered":"2024-09-11T05:19:36.773918+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:36.773918+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10064,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2950},{"shards":"0,2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13213","reported_seq":13362,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:36.741507+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:36.741507+0000","last_peered":"2024-09-11T05:19:36.741507+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:36.741507+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120809,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10220,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2863},{"shards":"0,2","objects":103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":427885185,"num_objects":104466,"num_object_clones":0,"num_object_copies":208932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35071,"num_objects_misplaced":114,"num_objects_unfound":0,"num_objects_dirty":104466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177229,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7327,"num_bytes_recovered":29990912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1024626688,"data_stored":1024548964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620684,"kb_used_data":323824,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93751156,"statfs":{"total":96636764160,"available":96001183744,"internally_reserved":0,"allocated":331595776,"data_stored":331326405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":303951531},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615068,"kb_used_data":308492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93756772,"statfs":{"total":96636764160,"available":96006934528,"internally_reserved":0,"allocated":315895808,"data_stored":315563217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215189,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752820,"kb_used_data":369948,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93619020,"statfs":{"total":96636764160,"available":95865876480,"internally_reserved":0,"allocated":378826752,"data_stored":378557376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378392576,"data_stored":378389097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315338752,"data_stored":315272058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330895360,"data_stored":330887809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:43.273 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:44.066 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:44.068 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:44.128 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-11T05:19:43.554152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419209876,"num_objects":102350,"num_object_clones":0,"num_object_copies":204700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179349,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7424,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1952576,"kb_used_data":991240,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":281162944,"statfs":{"total":289910292480,"available":287910854656,"internally_reserved":0,"allocated":1015029760,"data_stored":1014158422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":984337409},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,3,5,12,15,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27025408,"num_objects":-6598,"num_object_clones":0,"num_object_copies":-13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":101,"num_objects_misplaced":-375,"num_objects_unfound":0,"num_objects_dirty":-6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6598,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007750"},"pg_stats":[{"pgid":"2.f","version":"47'13788","reported_seq":13868,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.884751+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:41.884751+0000","last_peered":"2024-09-11T05:19:41.884751+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:41.884751+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26222592,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13510","reported_seq":13617,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.901681+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:41.901681+0000","last_peered":"2024-09-11T05:19:41.901681+0000","last_clean":"2024-09-11T05:19:41.901681+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:41.901681+0000","last_undegraded":"2024-09-11T05:19:41.901681+0000","last_fullsized":"2024-09-11T05:19:41.901681+0000","mapping_epoch":35,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26173440,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11908,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13370","reported_seq":14181,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.918243+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:41.918243+0000","last_peered":"2024-09-11T05:19:41.918243+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:41.918243+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997336,"num_objects":6348,"num_object_clones":0,"num_object_copies":12696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11773,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13630","reported_seq":13706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.869644+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:41.869644+0000","last_peered":"2024-09-11T05:19:41.869644+0000","last_clean":"2024-09-11T05:19:41.869644+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:41.869644+0000","last_undegraded":"2024-09-11T05:19:41.869644+0000","last_fullsized":"2024-09-11T05:19:41.869644+0000","mapping_epoch":35,"log_start":"40'3600","ondisk_log_start":"40'3600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13630,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13707","reported_seq":13842,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.834107+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:41.834107+0000","last_peered":"2024-09-11T05:19:41.834107+0000","last_clean":"2024-09-11T05:19:41.834107+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:41.834107+0000","last_undegraded":"2024-09-11T05:19:41.834107+0000","last_fullsized":"2024-09-11T05:19:41.834107+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26468352,"num_objects":6462,"num_object_clones":0,"num_object_copies":12924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10541,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13557","reported_seq":20908,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:38.990825+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:38.990825+0000","last_peered":"2024-09-11T05:19:38.990825+0000","last_clean":"2024-09-11T05:19:38.990825+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:38.990825+0000","last_undegraded":"2024-09-11T05:19:38.990825+0000","last_fullsized":"2024-09-11T05:19:38.990825+0000","mapping_epoch":46,"log_start":"39'3500","ondisk_log_start":"39'3500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26533888,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13688","reported_seq":13826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.831763+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:41.831763+0000","last_peered":"2024-09-11T05:19:41.831763+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:41.831763+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26681344,"num_objects":6514,"num_object_clones":0,"num_object_copies":13028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10853,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2938}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13466","reported_seq":13594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.846990+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:41.846990+0000","last_peered":"2024-09-11T05:19:41.846990+0000","last_clean":"2024-09-11T05:19:41.846990+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:41.846990+0000","last_undegraded":"2024-09-11T05:19:41.846990+0000","last_fullsized":"2024-09-11T05:19:41.846990+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10769,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13529","reported_seq":13665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.909693+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:41.909693+0000","last_peered":"2024-09-11T05:19:41.909693+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:41.909693+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11597,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13293","reported_seq":13463,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.750395+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:41.750395+0000","last_peered":"2024-09-11T05:19:41.750395+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:41.750395+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26128384,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10046,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2937}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13561","reported_seq":13684,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.897272+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:41.897272+0000","last_peered":"2024-09-11T05:19:41.897272+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:41.897272+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26509312,"num_objects":6472,"num_object_clones":0,"num_object_copies":12944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2950}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13350","reported_seq":20440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.890257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:41.890257+0000","last_peered":"2024-09-11T05:19:41.890257+0000","last_clean":"2024-09-11T05:19:41.890257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:41.890257+0000","last_undegraded":"2024-09-11T05:19:41.890257+0000","last_fullsized":"2024-09-11T05:19:41.890257+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26042368,"num_objects":6358,"num_object_clones":0,"num_object_copies":12716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11253,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13709","reported_seq":13846,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.863917+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:41.863917+0000","last_peered":"2024-09-11T05:19:41.863917+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:41.863917+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26357760,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10742,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13337","reported_seq":13476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.769277+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:41.769277+0000","last_peered":"2024-09-11T05:19:41.769277+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:41.769277+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25874432,"num_objects":6317,"num_object_clones":0,"num_object_copies":12634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2822}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13458","reported_seq":13598,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.806578+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:41.806578+0000","last_peered":"2024-09-11T05:19:41.806578+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:41.806578+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10216,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13330","reported_seq":13479,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.794138+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:41.794138+0000","last_peered":"2024-09-11T05:19:41.794138+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:41.794138+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25641577,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10337,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419209857,"num_objects":102348,"num_object_clones":0,"num_object_copies":204696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179347,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7423,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1013075968,"data_stored":1012998244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603804,"kb_used_data":319428,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93768036,"statfs":{"total":96636764160,"available":96018468864,"internally_reserved":0,"allocated":327094272,"data_stored":326824901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615068,"kb_used_data":308492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93756772,"statfs":{"total":96636764160,"available":96006934528,"internally_reserved":0,"allocated":315895808,"data_stored":315563217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215190,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733704,"kb_used_data":363320,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93638136,"statfs":{"total":96636764160,"available":95885451264,"internally_reserved":0,"allocated":372039680,"data_stored":371770304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379252395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,4,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371372032,"data_stored":371368553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315338752,"data_stored":315272058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326365184,"data_stored":326357633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:44.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:44.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13788", 'reported_seq': 13868, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.884751+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:41.884751+0000', 'last_peered': '2024-09-11T05:19:41.884751+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:41.884751+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26222592, 'num_objects': 6402, 'num_object_clones': 0, 'num_object_copies': 12804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13788, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2727}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:44.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:44.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13370", 'reported_seq': 14181, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.918243+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:19:41.918243+0000', 'last_peered': '2024-09-11T05:19:41.918243+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:41.918243+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25997336, 'num_objects': 6348, 'num_object_clones': 0, 'num_object_copies': 12696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6348, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11773, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 340, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3193}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:44.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:44.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13688", 'reported_seq': 13826, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.831763+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:41.831763+0000', 'last_peered': '2024-09-11T05:19:41.831763+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:41.831763+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'3600", 'ondisk_log_start': "43'3600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26681344, 'num_objects': 6514, 'num_object_clones': 0, 'num_object_copies': 13028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10853, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2938}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13529", 'reported_seq': 13665, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.909693+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:41.909693+0000', 'last_peered': '2024-09-11T05:19:41.909693+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:41.909693+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26386432, 'num_objects': 6442, 'num_object_clones': 0, 'num_object_copies': 12884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11597, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13293", 'reported_seq': 13463, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.750395+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:41.750395+0000', 'last_peered': '2024-09-11T05:19:41.750395+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:41.750395+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26128384, 'num_objects': 6379, 'num_object_clones': 0, 'num_object_copies': 12758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10046, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2937}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13561", 'reported_seq': 13684, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.897272+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:41.897272+0000', 'last_peered': '2024-09-11T05:19:41.897272+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:41.897272+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26509312, 'num_objects': 6472, 'num_object_clones': 0, 'num_object_copies': 12944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11590, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2950}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13709", 'reported_seq': 13846, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.863917+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:41.863917+0000', 'last_peered': '2024-09-11T05:19:41.863917+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:41.863917+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26357760, 'num_objects': 6435, 'num_object_clones': 0, 'num_object_copies': 12870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10742, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2812}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:44.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13337", 'reported_seq': 13476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.769277+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:41.769277+0000', 'last_peered': '2024-09-11T05:19:41.769277+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:41.769277+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25874432, 'num_objects': 6317, 'num_object_clones': 0, 'num_object_copies': 12634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6317, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10203, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2822}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:44.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:44.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13458", 'reported_seq': 13598, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.806578+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:41.806578+0000', 'last_peered': '2024-09-11T05:19:41.806578+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:41.806578+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25976832, 'num_objects': 6342, 'num_object_clones': 0, 'num_object_copies': 12684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10216, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:44.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:44.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13330", 'reported_seq': 13479, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:41.794138+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:41.794138+0000', 'last_peered': '2024-09-11T05:19:41.794138+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:41.794138+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25641577, 'num_objects': 6261, 'num_object_clones': 0, 'num_object_copies': 12522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10337, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2746}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:44.132 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:44.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:44.997 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35089/204700 objects degraded (17.142%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":462,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102350,"data_bytes":419209876,"bytes_used":1999437824,"bytes_avail":287910854656,"bytes_total":289910292480,"degraded_objects":35089,"degraded_total":204700,"degraded_ratio":0.17141670737664874,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":88007,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":1056768,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":549},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:45.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:45.057 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:45.806 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:45.866 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-11T05:19:45.554676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":418669204,"num_objects":102218,"num_object_clones":0,"num_object_copies":204436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102218,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179481,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7424,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1959152,"kb_used_data":988088,"kb_used_omap":12,"kb_used_meta":970995,"kb_avail":281156368,"statfs":{"total":289910292480,"available":287904120832,"internally_reserved":0,"allocated":1011802112,"data_stored":1010930774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":994298881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,2,5,10,17,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19042304,"num_objects":-4649,"num_object_clones":0,"num_object_copies":-9298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":-267,"num_objects_unfound":0,"num_objects_dirty":-4649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4649,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":675840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007833"},"pg_stats":[{"pgid":"2.f","version":"47'13788","reported_seq":13868,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.884751+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:41.884751+0000","last_peered":"2024-09-11T05:19:41.884751+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:41.884751+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26222592,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13510","reported_seq":13617,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.901681+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:41.901681+0000","last_peered":"2024-09-11T05:19:41.901681+0000","last_clean":"2024-09-11T05:19:41.901681+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:41.901681+0000","last_undegraded":"2024-09-11T05:19:41.901681+0000","last_fullsized":"2024-09-11T05:19:41.901681+0000","mapping_epoch":35,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26173440,"num_objects":6390,"num_object_clones":0,"num_object_copies":12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11908,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13370","reported_seq":14181,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.918243+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:41.918243+0000","last_peered":"2024-09-11T05:19:41.918243+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:41.918243+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997336,"num_objects":6348,"num_object_clones":0,"num_object_copies":12696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11773,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13630","reported_seq":13706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.869644+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:41.869644+0000","last_peered":"2024-09-11T05:19:41.869644+0000","last_clean":"2024-09-11T05:19:41.869644+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:41.869644+0000","last_undegraded":"2024-09-11T05:19:41.869644+0000","last_fullsized":"2024-09-11T05:19:41.869644+0000","mapping_epoch":35,"log_start":"40'3600","ondisk_log_start":"40'3600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13630,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13707","reported_seq":13842,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.834107+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:41.834107+0000","last_peered":"2024-09-11T05:19:41.834107+0000","last_clean":"2024-09-11T05:19:41.834107+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:41.834107+0000","last_undegraded":"2024-09-11T05:19:41.834107+0000","last_fullsized":"2024-09-11T05:19:41.834107+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26468352,"num_objects":6462,"num_object_clones":0,"num_object_copies":12924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10541,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13689","reported_seq":21040,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:44.027736+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:44.027736+0000","last_peered":"2024-09-11T05:19:44.027736+0000","last_clean":"2024-09-11T05:19:44.027736+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:44.027736+0000","last_undegraded":"2024-09-11T05:19:44.027736+0000","last_fullsized":"2024-09-11T05:19:44.027736+0000","mapping_epoch":46,"log_start":"40'3600","ondisk_log_start":"40'3600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25993216,"num_objects":6346,"num_object_clones":0,"num_object_copies":12692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10233,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13688","reported_seq":13826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.831763+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:41.831763+0000","last_peered":"2024-09-11T05:19:41.831763+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:41.831763+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26681344,"num_objects":6514,"num_object_clones":0,"num_object_copies":13028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10853,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2938}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13466","reported_seq":13594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.846990+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:41.846990+0000","last_peered":"2024-09-11T05:19:41.846990+0000","last_clean":"2024-09-11T05:19:41.846990+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:41.846990+0000","last_undegraded":"2024-09-11T05:19:41.846990+0000","last_fullsized":"2024-09-11T05:19:41.846990+0000","mapping_epoch":34,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10769,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13529","reported_seq":13665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.909693+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:41.909693+0000","last_peered":"2024-09-11T05:19:41.909693+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:41.909693+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11597,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13293","reported_seq":13463,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.750395+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:41.750395+0000","last_peered":"2024-09-11T05:19:41.750395+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:41.750395+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"39'3200","ondisk_log_start":"39'3200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26128384,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10046,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2937}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13561","reported_seq":13684,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.897272+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:41.897272+0000","last_peered":"2024-09-11T05:19:41.897272+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:41.897272+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26509312,"num_objects":6472,"num_object_clones":0,"num_object_copies":12944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2950}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13350","reported_seq":20440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:41.890257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:41.890257+0000","last_peered":"2024-09-11T05:19:41.890257+0000","last_clean":"2024-09-11T05:19:41.890257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:41.890257+0000","last_undegraded":"2024-09-11T05:19:41.890257+0000","last_fullsized":"2024-09-11T05:19:41.890257+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26042368,"num_objects":6358,"num_object_clones":0,"num_object_copies":12716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11253,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13709","reported_seq":13846,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.863917+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:41.863917+0000","last_peered":"2024-09-11T05:19:41.863917+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:41.863917+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26357760,"num_objects":6435,"num_object_clones":0,"num_object_copies":12870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10742,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13337","reported_seq":13476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.769277+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:41.769277+0000","last_peered":"2024-09-11T05:19:41.769277+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:41.769277+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25874432,"num_objects":6317,"num_object_clones":0,"num_object_copies":12634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2822}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13458","reported_seq":13598,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.806578+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:41.806578+0000","last_peered":"2024-09-11T05:19:41.806578+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:41.806578+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10216,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13330","reported_seq":13479,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:41.794138+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:41.794138+0000","last_peered":"2024-09-11T05:19:41.794138+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:41.794138+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"39'3300","ondisk_log_start":"39'3300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25641577,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10337,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":418669185,"num_objects":102216,"num_object_clones":0,"num_object_copies":204432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179479,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7423,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1008820224,"data_stored":1008742500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603804,"kb_used_data":319428,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93768036,"statfs":{"total":96636764160,"available":96018468864,"internally_reserved":0,"allocated":327094272,"data_stored":326824901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621644,"kb_used_data":305340,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93750196,"statfs":{"total":96636764160,"available":96000200704,"internally_reserved":0,"allocated":312668160,"data_stored":312335569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215190,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733704,"kb_used_data":363320,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93638136,"statfs":{"total":96636764160,"available":95885451264,"internally_reserved":0,"allocated":372039680,"data_stored":371770304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379252395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,4,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371372032,"data_stored":371368553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311083008,"data_stored":311016314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326365184,"data_stored":326357633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:48.868 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:49.619 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:49.625 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:49.674 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-11T05:19:49.556219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409461396,"num_objects":99970,"num_object_clones":0,"num_object_copies":199940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181729,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7498,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1953680,"kb_used_data":970136,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281161840,"statfs":{"total":289910292480,"available":287909724160,"internally_reserved":0,"allocated":993419264,"data_stored":992547926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1007078401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,6,6,7,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19083264,"num_objects":-4659,"num_object_clones":0,"num_object_copies":-9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-56,"num_objects_misplaced":-114,"num_objects_unfound":0,"num_objects_dirty":-4659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4659,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"2.f","version":"47'13920","reported_seq":14004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.806969+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:46.806969+0000","last_peered":"2024-09-11T05:19:46.806969+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:46.806969+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13920,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13637","reported_seq":13744,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.908216+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:46.908216+0000","last_peered":"2024-09-11T05:19:46.908216+0000","last_clean":"2024-09-11T05:19:46.908216+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:46.908216+0000","last_undegraded":"2024-09-11T05:19:46.908216+0000","last_fullsized":"2024-09-11T05:19:46.908216+0000","mapping_epoch":35,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13516","reported_seq":14476,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.956086+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:46.956086+0000","last_peered":"2024-09-11T05:19:46.956086+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:46.956086+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399320,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11919,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13764","reported_seq":13840,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.849679+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:46.849679+0000","last_peered":"2024-09-11T05:19:46.849679+0000","last_clean":"2024-09-11T05:19:46.849679+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:46.849679+0000","last_undegraded":"2024-09-11T05:19:46.849679+0000","last_fullsized":"2024-09-11T05:19:46.849679+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25378816,"num_objects":6196,"num_object_clones":0,"num_object_copies":12392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13764,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13849","reported_seq":13984,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.720279+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:46.720279+0000","last_peered":"2024-09-11T05:19:46.720279+0000","last_clean":"2024-09-11T05:19:46.720279+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:46.720279+0000","last_undegraded":"2024-09-11T05:19:46.720279+0000","last_fullsized":"2024-09-11T05:19:46.720279+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10683,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13840","reported_seq":21191,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:48.920538+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:48.920538+0000","last_peered":"2024-09-11T05:19:48.920538+0000","last_clean":"2024-09-11T05:19:48.920538+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:48.920538+0000","last_undegraded":"2024-09-11T05:19:48.920538+0000","last_fullsized":"2024-09-11T05:19:48.920538+0000","mapping_epoch":46,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25374720,"num_objects":6195,"num_object_clones":0,"num_object_copies":12390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10384,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13838","reported_seq":13976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.786614+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:46.786614+0000","last_peered":"2024-09-11T05:19:46.786614+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:46.786614+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11003,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13599","reported_seq":13728,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.876252+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:46.876252+0000","last_peered":"2024-09-11T05:19:46.876252+0000","last_clean":"2024-09-11T05:19:46.876252+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:46.876252+0000","last_undegraded":"2024-09-11T05:19:46.876252+0000","last_fullsized":"2024-09-11T05:19:46.876252+0000","mapping_epoch":34,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25743360,"num_objects":6285,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10902,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13671","reported_seq":13808,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.925772+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:46.925772+0000","last_peered":"2024-09-11T05:19:46.925772+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:46.925772+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25804800,"num_objects":6300,"num_object_clones":0,"num_object_copies":12600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13434","reported_seq":13604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.877059+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:46.877059+0000","last_peered":"2024-09-11T05:19:46.877059+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:46.877059+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"40'3400","ondisk_log_start":"40'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25550848,"num_objects":6238,"num_object_clones":0,"num_object_copies":12476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10187,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13686","reported_seq":13809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.873117+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:46.873117+0000","last_peered":"2024-09-11T05:19:46.873117+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:46.873117+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13514","reported_seq":20604,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.873740+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:46.873740+0000","last_peered":"2024-09-11T05:19:46.873740+0000","last_clean":"2024-09-11T05:19:46.873740+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:46.873740+0000","last_undegraded":"2024-09-11T05:19:46.873740+0000","last_fullsized":"2024-09-11T05:19:46.873740+0000","mapping_epoch":44,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25370624,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11417,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13861","reported_seq":13995,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.842800+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:46.842800+0000","last_peered":"2024-09-11T05:19:46.842800+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:46.842800+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25735168,"num_objects":6283,"num_object_clones":0,"num_object_copies":12566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13464","reported_seq":13603,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.846589+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:46.846589+0000","last_peered":"2024-09-11T05:19:46.846589+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:46.846589+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"40'3400","ondisk_log_start":"40'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25354240,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10330,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2695}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13596","reported_seq":13736,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.869763+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:46.869763+0000","last_peered":"2024-09-11T05:19:46.869763+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:46.869763+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25411584,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10354,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13474","reported_seq":13623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.858583+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:46.858583+0000","last_peered":"2024-09-11T05:19:46.858583+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:46.858583+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051753,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10481,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409461377,"num_objects":99968,"num_object_clones":0,"num_object_copies":199936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181727,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7497,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":990392320,"data_stored":990314596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603584,"kb_used_data":315056,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93768256,"statfs":{"total":96636764160,"available":96018694144,"internally_reserved":0,"allocated":322617344,"data_stored":322347973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619008,"kb_used_data":298540,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93752832,"statfs":{"total":96636764160,"available":96002899968,"internally_reserved":0,"allocated":305704960,"data_stored":305372369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215191,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731088,"kb_used_data":356540,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93640752,"statfs":{"total":96636764160,"available":95888130048,"internally_reserved":0,"allocated":365096960,"data_stored":364827584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364535808,"data_stored":364532329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304050176,"data_stored":303983482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321806336,"data_stored":321798785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:49.675 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:50.498 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:50.706 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-11T05:19:49.556219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409461396,"num_objects":99970,"num_object_clones":0,"num_object_copies":199940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181729,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7498,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1953680,"kb_used_data":970136,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281161840,"statfs":{"total":289910292480,"available":287909724160,"internally_reserved":0,"allocated":993419264,"data_stored":992547926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1007078401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,6,6,7,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19083264,"num_objects":-4659,"num_object_clones":0,"num_object_copies":-9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-56,"num_objects_misplaced":-114,"num_objects_unfound":0,"num_objects_dirty":-4659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4659,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"2.f","version":"47'13920","reported_seq":14004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.806969+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:46.806969+0000","last_peered":"2024-09-11T05:19:46.806969+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:46.806969+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13920,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13637","reported_seq":13744,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.908216+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:46.908216+0000","last_peered":"2024-09-11T05:19:46.908216+0000","last_clean":"2024-09-11T05:19:46.908216+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:46.908216+0000","last_undegraded":"2024-09-11T05:19:46.908216+0000","last_fullsized":"2024-09-11T05:19:46.908216+0000","mapping_epoch":35,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13516","reported_seq":14476,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.956086+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:46.956086+0000","last_peered":"2024-09-11T05:19:46.956086+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:46.956086+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399320,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11919,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13764","reported_seq":13840,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.849679+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:46.849679+0000","last_peered":"2024-09-11T05:19:46.849679+0000","last_clean":"2024-09-11T05:19:46.849679+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:46.849679+0000","last_undegraded":"2024-09-11T05:19:46.849679+0000","last_fullsized":"2024-09-11T05:19:46.849679+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25378816,"num_objects":6196,"num_object_clones":0,"num_object_copies":12392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13764,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13849","reported_seq":13984,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.720279+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:46.720279+0000","last_peered":"2024-09-11T05:19:46.720279+0000","last_clean":"2024-09-11T05:19:46.720279+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:46.720279+0000","last_undegraded":"2024-09-11T05:19:46.720279+0000","last_fullsized":"2024-09-11T05:19:46.720279+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10683,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13840","reported_seq":21191,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:48.920538+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:48.920538+0000","last_peered":"2024-09-11T05:19:48.920538+0000","last_clean":"2024-09-11T05:19:48.920538+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:48.920538+0000","last_undegraded":"2024-09-11T05:19:48.920538+0000","last_fullsized":"2024-09-11T05:19:48.920538+0000","mapping_epoch":46,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25374720,"num_objects":6195,"num_object_clones":0,"num_object_copies":12390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10384,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13838","reported_seq":13976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.786614+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:46.786614+0000","last_peered":"2024-09-11T05:19:46.786614+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:46.786614+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11003,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13599","reported_seq":13728,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.876252+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:46.876252+0000","last_peered":"2024-09-11T05:19:46.876252+0000","last_clean":"2024-09-11T05:19:46.876252+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:46.876252+0000","last_undegraded":"2024-09-11T05:19:46.876252+0000","last_fullsized":"2024-09-11T05:19:46.876252+0000","mapping_epoch":34,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25743360,"num_objects":6285,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10902,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13671","reported_seq":13808,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.925772+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:46.925772+0000","last_peered":"2024-09-11T05:19:46.925772+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:46.925772+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25804800,"num_objects":6300,"num_object_clones":0,"num_object_copies":12600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13434","reported_seq":13604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.877059+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:46.877059+0000","last_peered":"2024-09-11T05:19:46.877059+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:46.877059+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"40'3400","ondisk_log_start":"40'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25550848,"num_objects":6238,"num_object_clones":0,"num_object_copies":12476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10187,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13686","reported_seq":13809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.873117+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:46.873117+0000","last_peered":"2024-09-11T05:19:46.873117+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:46.873117+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13514","reported_seq":20604,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.873740+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:46.873740+0000","last_peered":"2024-09-11T05:19:46.873740+0000","last_clean":"2024-09-11T05:19:46.873740+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:46.873740+0000","last_undegraded":"2024-09-11T05:19:46.873740+0000","last_fullsized":"2024-09-11T05:19:46.873740+0000","mapping_epoch":44,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25370624,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11417,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13861","reported_seq":13995,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.842800+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:46.842800+0000","last_peered":"2024-09-11T05:19:46.842800+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:46.842800+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25735168,"num_objects":6283,"num_object_clones":0,"num_object_copies":12566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13464","reported_seq":13603,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.846589+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:46.846589+0000","last_peered":"2024-09-11T05:19:46.846589+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:46.846589+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"40'3400","ondisk_log_start":"40'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25354240,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10330,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2695}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13596","reported_seq":13736,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.869763+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:46.869763+0000","last_peered":"2024-09-11T05:19:46.869763+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:46.869763+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25411584,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10354,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13474","reported_seq":13623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.858583+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:46.858583+0000","last_peered":"2024-09-11T05:19:46.858583+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:46.858583+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051753,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10481,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409461377,"num_objects":99968,"num_object_clones":0,"num_object_copies":199936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181727,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7497,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":990392320,"data_stored":990314596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603584,"kb_used_data":315056,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93768256,"statfs":{"total":96636764160,"available":96018694144,"internally_reserved":0,"allocated":322617344,"data_stored":322347973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619008,"kb_used_data":298540,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93752832,"statfs":{"total":96636764160,"available":96002899968,"internally_reserved":0,"allocated":305704960,"data_stored":305372369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215191,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731088,"kb_used_data":356540,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93640752,"statfs":{"total":96636764160,"available":95888130048,"internally_reserved":0,"allocated":365096960,"data_stored":364827584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364535808,"data_stored":364532329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304050176,"data_stored":303983482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321806336,"data_stored":321798785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13920", 'reported_seq': 14004, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.806969+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:46.806969+0000', 'last_peered': '2024-09-11T05:19:46.806969+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:46.806969+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25681920, 'num_objects': 6270, 'num_object_clones': 0, 'num_object_copies': 12540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13920, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2595}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13516", 'reported_seq': 14476, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.956086+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:19:46.956086+0000', 'last_peered': '2024-09-11T05:19:46.956086+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:46.956086+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'3500", 'ondisk_log_start': "43'3500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25399320, 'num_objects': 6202, 'num_object_clones': 0, 'num_object_copies': 12404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3081, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6202, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11919, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 414, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13838", 'reported_seq': 13976, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.786614+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:46.786614+0000', 'last_peered': '2024-09-11T05:19:46.786614+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:46.786614+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26066944, 'num_objects': 6364, 'num_object_clones': 0, 'num_object_copies': 12728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11003, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2788}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13671", 'reported_seq': 13808, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.925772+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:46.925772+0000', 'last_peered': '2024-09-11T05:19:46.925772+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:46.925772+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'3600", 'ondisk_log_start': "43'3600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25804800, 'num_objects': 6300, 'num_object_clones': 0, 'num_object_copies': 12600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11739, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2775}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:50.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13434", 'reported_seq': 13604, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.877059+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:46.877059+0000', 'last_peered': '2024-09-11T05:19:46.877059+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:46.877059+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "40'3400", 'ondisk_log_start': "40'3400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25550848, 'num_objects': 6238, 'num_object_clones': 0, 'num_object_copies': 12476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10187, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2796}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13686", 'reported_seq': 13809, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.873117+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:46.873117+0000', 'last_peered': '2024-09-11T05:19:46.873117+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:46.873117+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'3600", 'ondisk_log_start': "43'3600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25997312, 'num_objects': 6347, 'num_object_clones': 0, 'num_object_copies': 12694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11715, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2825}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13861", 'reported_seq': 13995, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.842800+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:46.842800+0000', 'last_peered': '2024-09-11T05:19:46.842800+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:46.842800+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25735168, 'num_objects': 6283, 'num_object_clones': 0, 'num_object_copies': 12566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10894, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2660}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13464", 'reported_seq': 13603, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.846589+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:46.846589+0000', 'last_peered': '2024-09-11T05:19:46.846589+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:46.846589+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "40'3400", 'ondisk_log_start': "40'3400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25354240, 'num_objects': 6190, 'num_object_clones': 0, 'num_object_copies': 12380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10330, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2695}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13596", 'reported_seq': 13736, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.869763+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:46.869763+0000', 'last_peered': '2024-09-11T05:19:46.869763+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:46.869763+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25411584, 'num_objects': 6204, 'num_object_clones': 0, 'num_object_copies': 12408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10354, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2660}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:50.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:50.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13474", 'reported_seq': 13623, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:46.858583+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:46.858583+0000', 'last_peered': '2024-09-11T05:19:46.858583+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:46.858583+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "39'3400", 'ondisk_log_start': "39'3400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25051753, 'num_objects': 6117, 'num_object_clones': 0, 'num_object_copies': 12234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10481, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2602}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:50.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:51.861 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:51.862 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35015/199940 objects degraded (17.513%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":469,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":99970,"data_bytes":409461396,"bytes_used":2000568320,"bytes_avail":287909724160,"bytes_total":289910292480,"degraded_objects":35015,"degraded_total":199940,"degraded_ratio":0.17512753826147842,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":27297,"recovering_keys_per_sec":0,"num_objects_recovered":170,"num_bytes_recovered":327680,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":374},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:51.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:51.942 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:52.792 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:52.793 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:52.841 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-11T05:19:51.557316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409461396,"num_objects":99970,"num_object_clones":0,"num_object_copies":199940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181729,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7498,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1953680,"kb_used_data":970136,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281161840,"statfs":{"total":289910292480,"available":287909724160,"internally_reserved":0,"allocated":993419264,"data_stored":992547926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1007078401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,6,6,7,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18423808,"num_objects":-4498,"num_object_clones":0,"num_object_copies":-8996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-56,"num_objects_misplaced":-114,"num_objects_unfound":0,"num_objects_dirty":-4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004148"},"pg_stats":[{"pgid":"2.f","version":"47'13920","reported_seq":14004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.806969+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:46.806969+0000","last_peered":"2024-09-11T05:19:46.806969+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:46.806969+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13920,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13637","reported_seq":13744,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.908216+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:46.908216+0000","last_peered":"2024-09-11T05:19:46.908216+0000","last_clean":"2024-09-11T05:19:46.908216+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:46.908216+0000","last_undegraded":"2024-09-11T05:19:46.908216+0000","last_fullsized":"2024-09-11T05:19:46.908216+0000","mapping_epoch":35,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13516","reported_seq":14476,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.956086+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:46.956086+0000","last_peered":"2024-09-11T05:19:46.956086+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:46.956086+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399320,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11919,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13764","reported_seq":13840,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.849679+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:46.849679+0000","last_peered":"2024-09-11T05:19:46.849679+0000","last_clean":"2024-09-11T05:19:46.849679+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:46.849679+0000","last_undegraded":"2024-09-11T05:19:46.849679+0000","last_fullsized":"2024-09-11T05:19:46.849679+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25378816,"num_objects":6196,"num_object_clones":0,"num_object_copies":12392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13764,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13849","reported_seq":13984,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.720279+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:46.720279+0000","last_peered":"2024-09-11T05:19:46.720279+0000","last_clean":"2024-09-11T05:19:46.720279+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:46.720279+0000","last_undegraded":"2024-09-11T05:19:46.720279+0000","last_fullsized":"2024-09-11T05:19:46.720279+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10683,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13840","reported_seq":21191,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:48.920538+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:48.920538+0000","last_peered":"2024-09-11T05:19:48.920538+0000","last_clean":"2024-09-11T05:19:48.920538+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:48.920538+0000","last_undegraded":"2024-09-11T05:19:48.920538+0000","last_fullsized":"2024-09-11T05:19:48.920538+0000","mapping_epoch":46,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25374720,"num_objects":6195,"num_object_clones":0,"num_object_copies":12390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10384,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13838","reported_seq":13976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.786614+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:46.786614+0000","last_peered":"2024-09-11T05:19:46.786614+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:46.786614+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11003,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13599","reported_seq":13728,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.876252+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:46.876252+0000","last_peered":"2024-09-11T05:19:46.876252+0000","last_clean":"2024-09-11T05:19:46.876252+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:46.876252+0000","last_undegraded":"2024-09-11T05:19:46.876252+0000","last_fullsized":"2024-09-11T05:19:46.876252+0000","mapping_epoch":34,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25743360,"num_objects":6285,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10902,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13671","reported_seq":13808,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.925772+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:46.925772+0000","last_peered":"2024-09-11T05:19:46.925772+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:46.925772+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25804800,"num_objects":6300,"num_object_clones":0,"num_object_copies":12600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13434","reported_seq":13604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.877059+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:46.877059+0000","last_peered":"2024-09-11T05:19:46.877059+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:46.877059+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"40'3400","ondisk_log_start":"40'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25550848,"num_objects":6238,"num_object_clones":0,"num_object_copies":12476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10187,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13686","reported_seq":13809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.873117+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:46.873117+0000","last_peered":"2024-09-11T05:19:46.873117+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:46.873117+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2825}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13514","reported_seq":20604,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:46.873740+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:46.873740+0000","last_peered":"2024-09-11T05:19:46.873740+0000","last_clean":"2024-09-11T05:19:46.873740+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:46.873740+0000","last_undegraded":"2024-09-11T05:19:46.873740+0000","last_fullsized":"2024-09-11T05:19:46.873740+0000","mapping_epoch":44,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25370624,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11417,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13861","reported_seq":13995,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.842800+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:46.842800+0000","last_peered":"2024-09-11T05:19:46.842800+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:46.842800+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25735168,"num_objects":6283,"num_object_clones":0,"num_object_copies":12566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13464","reported_seq":13603,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.846589+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:46.846589+0000","last_peered":"2024-09-11T05:19:46.846589+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:46.846589+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"40'3400","ondisk_log_start":"40'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25354240,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10330,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2695}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13596","reported_seq":13736,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.869763+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:46.869763+0000","last_peered":"2024-09-11T05:19:46.869763+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:46.869763+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"40'3500","ondisk_log_start":"40'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25411584,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10354,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13474","reported_seq":13623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:46.858583+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:46.858583+0000","last_peered":"2024-09-11T05:19:46.858583+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:46.858583+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"39'3400","ondisk_log_start":"39'3400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051753,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10481,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409461377,"num_objects":99968,"num_object_clones":0,"num_object_copies":199936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35015,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181727,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7497,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":990392320,"data_stored":990314596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603584,"kb_used_data":315056,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93768256,"statfs":{"total":96636764160,"available":96018694144,"internally_reserved":0,"allocated":322617344,"data_stored":322347973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619008,"kb_used_data":298540,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93752832,"statfs":{"total":96636764160,"available":96002899968,"internally_reserved":0,"allocated":305704960,"data_stored":305372369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215191,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731088,"kb_used_data":356540,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93640752,"statfs":{"total":96636764160,"available":95888130048,"internally_reserved":0,"allocated":365096960,"data_stored":364827584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364535808,"data_stored":364532329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304050176,"data_stored":303983482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321806336,"data_stored":321798785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:55.843 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:56.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:56.534 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:56.581 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-11T05:19:55.559918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":400200340,"num_objects":97709,"num_object_clones":0,"num_object_copies":195418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183990,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7573,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160727,"ondisk_log_size":160727,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1956696,"kb_used_data":952352,"kb_used_omap":12,"kb_used_meta":1004275,"kb_avail":281158824,"statfs":{"total":289910292480,"available":287906635776,"internally_reserved":0,"allocated":975208448,"data_stored":974337110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1028377601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,10,11,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19009536,"num_objects":-4641,"num_object_clones":0,"num_object_copies":-9282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4641,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005765"},"pg_stats":[{"pgid":"2.f","version":"47'14066","reported_seq":14150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.784273+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:51.784273+0000","last_peered":"2024-09-11T05:19:51.784273+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:51.784273+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14066,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13787","reported_seq":13894,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.935482+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:51.935482+0000","last_peered":"2024-09-11T05:19:51.935482+0000","last_clean":"2024-09-11T05:19:51.935482+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:51.935482+0000","last_undegraded":"2024-09-11T05:19:51.935482+0000","last_fullsized":"2024-09-11T05:19:51.935482+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12185,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13654","reported_seq":14760,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.911433+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:51.911433+0000","last_peered":"2024-09-11T05:19:51.911433+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:51.911433+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24834072,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12057,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13929","reported_seq":14005,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.956595+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:51.956595+0000","last_peered":"2024-09-11T05:19:51.956595+0000","last_clean":"2024-09-11T05:19:51.956595+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:51.956595+0000","last_undegraded":"2024-09-11T05:19:51.956595+0000","last_fullsized":"2024-09-11T05:19:51.956595+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24702976,"num_objects":6031,"num_object_clones":0,"num_object_copies":12062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13929,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13998","reported_seq":14133,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.847416+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:51.847416+0000","last_peered":"2024-09-11T05:19:51.847416+0000","last_clean":"2024-09-11T05:19:51.847416+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:51.847416+0000","last_undegraded":"2024-09-11T05:19:51.847416+0000","last_fullsized":"2024-09-11T05:19:51.847416+0000","mapping_epoch":34,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25276416,"num_objects":6171,"num_object_clones":0,"num_object_copies":12342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10832,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13967","reported_seq":21318,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:54.107294+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:54.107294+0000","last_peered":"2024-09-11T05:19:54.107294+0000","last_clean":"2024-09-11T05:19:54.107294+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:54.107294+0000","last_undegraded":"2024-09-11T05:19:54.107294+0000","last_fullsized":"2024-09-11T05:19:54.107294+0000","mapping_epoch":46,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24854528,"num_objects":6068,"num_object_clones":0,"num_object_copies":12136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10511,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13963","reported_seq":14100,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.783915+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:51.783915+0000","last_peered":"2024-09-11T05:19:51.783915+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:51.783915+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25554944,"num_objects":6239,"num_object_clones":0,"num_object_copies":12478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11128,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13744","reported_seq":13873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.748193+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:51.748193+0000","last_peered":"2024-09-11T05:19:51.748193+0000","last_clean":"2024-09-11T05:19:51.748193+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:51.748193+0000","last_undegraded":"2024-09-11T05:19:51.748193+0000","last_fullsized":"2024-09-11T05:19:51.748193+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13814","reported_seq":13951,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.973041+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:51.973041+0000","last_peered":"2024-09-11T05:19:51.973041+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:51.973041+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11882,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13555","reported_seq":13725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.840990+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:51.840990+0000","last_peered":"2024-09-11T05:19:51.840990+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:51.840990+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10308,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13834","reported_seq":13957,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.939274+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:51.939274+0000","last_peered":"2024-09-11T05:19:51.939274+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:51.939274+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2677}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13662","reported_seq":20752,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.842438+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:51.842438+0000","last_peered":"2024-09-11T05:19:51.842438+0000","last_clean":"2024-09-11T05:19:51.842438+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:51.842438+0000","last_undegraded":"2024-09-11T05:19:51.842438+0000","last_fullsized":"2024-09-11T05:19:51.842438+0000","mapping_epoch":44,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24764416,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11565,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14004","reported_seq":14140,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.892456+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:51.892456+0000","last_peered":"2024-09-11T05:19:51.892456+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:51.892456+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11037,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13603","reported_seq":13742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.752480+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:51.752480+0000","last_peered":"2024-09-11T05:19:51.752480+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:51.752480+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24784896,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10469,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13738","reported_seq":13878,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.888601+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:51.888601+0000","last_peered":"2024-09-11T05:19:51.888601+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:51.888601+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24829952,"num_objects":6062,"num_object_clones":0,"num_object_copies":12124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2518}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13606","reported_seq":13755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.878867+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:51.878867+0000","last_peered":"2024-09-11T05:19:51.878867+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:51.878867+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24511081,"num_objects":5985,"num_object_clones":0,"num_object_copies":11970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10613,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":400200321,"num_objects":97707,"num_object_clones":0,"num_object_copies":195414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183988,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7572,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":971870208,"data_stored":971792484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603112,"kb_used_data":310424,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93768728,"statfs":{"total":96636764160,"available":96019177472,"internally_reserved":0,"allocated":317874176,"data_stored":317604805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,5,5,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620868,"kb_used_data":292084,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750972,"statfs":{"total":96636764160,"available":96000995328,"internally_reserved":0,"allocated":299094016,"data_stored":298761425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215192,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732716,"kb_used_data":349844,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93639124,"statfs":{"total":96636764160,"available":95886462976,"internally_reserved":0,"allocated":358240256,"data_stored":357970880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,6,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357687296,"data_stored":357683817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297246720,"data_stored":297180026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316936192,"data_stored":316928641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:56.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:57.451 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:57.452 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:57.485 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-11T05:19:55.559918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":400200340,"num_objects":97709,"num_object_clones":0,"num_object_copies":195418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183990,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7573,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160727,"ondisk_log_size":160727,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1956696,"kb_used_data":952352,"kb_used_omap":12,"kb_used_meta":1004275,"kb_avail":281158824,"statfs":{"total":289910292480,"available":287906635776,"internally_reserved":0,"allocated":975208448,"data_stored":974337110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1028377601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,10,11,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19009536,"num_objects":-4641,"num_object_clones":0,"num_object_copies":-9282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4641,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005765"},"pg_stats":[{"pgid":"2.f","version":"47'14066","reported_seq":14150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.784273+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:51.784273+0000","last_peered":"2024-09-11T05:19:51.784273+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:51.784273+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14066,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13787","reported_seq":13894,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.935482+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:51.935482+0000","last_peered":"2024-09-11T05:19:51.935482+0000","last_clean":"2024-09-11T05:19:51.935482+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:51.935482+0000","last_undegraded":"2024-09-11T05:19:51.935482+0000","last_fullsized":"2024-09-11T05:19:51.935482+0000","mapping_epoch":35,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12185,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13654","reported_seq":14760,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.911433+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:51.911433+0000","last_peered":"2024-09-11T05:19:51.911433+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:51.911433+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24834072,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12057,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13929","reported_seq":14005,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.956595+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:51.956595+0000","last_peered":"2024-09-11T05:19:51.956595+0000","last_clean":"2024-09-11T05:19:51.956595+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:51.956595+0000","last_undegraded":"2024-09-11T05:19:51.956595+0000","last_fullsized":"2024-09-11T05:19:51.956595+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24702976,"num_objects":6031,"num_object_clones":0,"num_object_copies":12062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13929,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13998","reported_seq":14133,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.847416+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:51.847416+0000","last_peered":"2024-09-11T05:19:51.847416+0000","last_clean":"2024-09-11T05:19:51.847416+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:51.847416+0000","last_undegraded":"2024-09-11T05:19:51.847416+0000","last_fullsized":"2024-09-11T05:19:51.847416+0000","mapping_epoch":34,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25276416,"num_objects":6171,"num_object_clones":0,"num_object_copies":12342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10832,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13967","reported_seq":21318,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:54.107294+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:54.107294+0000","last_peered":"2024-09-11T05:19:54.107294+0000","last_clean":"2024-09-11T05:19:54.107294+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:54.107294+0000","last_undegraded":"2024-09-11T05:19:54.107294+0000","last_fullsized":"2024-09-11T05:19:54.107294+0000","mapping_epoch":46,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24854528,"num_objects":6068,"num_object_clones":0,"num_object_copies":12136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10511,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'13963","reported_seq":14100,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.783915+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:51.783915+0000","last_peered":"2024-09-11T05:19:51.783915+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:51.783915+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25554944,"num_objects":6239,"num_object_clones":0,"num_object_copies":12478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11128,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13744","reported_seq":13873,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.748193+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:51.748193+0000","last_peered":"2024-09-11T05:19:51.748193+0000","last_clean":"2024-09-11T05:19:51.748193+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:51.748193+0000","last_undegraded":"2024-09-11T05:19:51.748193+0000","last_fullsized":"2024-09-11T05:19:51.748193+0000","mapping_epoch":34,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13814","reported_seq":13951,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.973041+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:51.973041+0000","last_peered":"2024-09-11T05:19:51.973041+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:51.973041+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11882,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13555","reported_seq":13725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.840990+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:51.840990+0000","last_peered":"2024-09-11T05:19:51.840990+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:51.840990+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3500","ondisk_log_start":"43'3500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10308,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13834","reported_seq":13957,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.939274+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:51.939274+0000","last_peered":"2024-09-11T05:19:51.939274+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:51.939274+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2677}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13662","reported_seq":20752,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:51.842438+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:51.842438+0000","last_peered":"2024-09-11T05:19:51.842438+0000","last_clean":"2024-09-11T05:19:51.842438+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:51.842438+0000","last_undegraded":"2024-09-11T05:19:51.842438+0000","last_fullsized":"2024-09-11T05:19:51.842438+0000","mapping_epoch":44,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24764416,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11565,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14004","reported_seq":14140,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.892456+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:51.892456+0000","last_peered":"2024-09-11T05:19:51.892456+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:51.892456+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11037,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13603","reported_seq":13742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.752480+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:51.752480+0000","last_peered":"2024-09-11T05:19:51.752480+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:51.752480+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24784896,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10469,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13738","reported_seq":13878,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.888601+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:51.888601+0000","last_peered":"2024-09-11T05:19:51.888601+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:51.888601+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24829952,"num_objects":6062,"num_object_clones":0,"num_object_copies":12124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2518}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13606","reported_seq":13755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:51.878867+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:51.878867+0000","last_peered":"2024-09-11T05:19:51.878867+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:51.878867+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'3600","ondisk_log_start":"43'3600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24511081,"num_objects":5985,"num_object_clones":0,"num_object_copies":11970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10613,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":400200321,"num_objects":97707,"num_object_clones":0,"num_object_copies":195414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183988,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7572,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":971870208,"data_stored":971792484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603112,"kb_used_data":310424,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93768728,"statfs":{"total":96636764160,"available":96019177472,"internally_reserved":0,"allocated":317874176,"data_stored":317604805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299691691},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,5,5,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620868,"kb_used_data":292084,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750972,"statfs":{"total":96636764160,"available":96000995328,"internally_reserved":0,"allocated":299094016,"data_stored":298761425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215192,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732716,"kb_used_data":349844,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93639124,"statfs":{"total":96636764160,"available":95886462976,"internally_reserved":0,"allocated":358240256,"data_stored":357970880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,6,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357687296,"data_stored":357683817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297246720,"data_stored":297180026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316936192,"data_stored":316928641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:19:57.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:19:57.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14066", 'reported_seq': 14150, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.784273+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:19:51.784273+0000', 'last_peered': '2024-09-11T05:19:51.784273+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:19:51.784273+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25083904, 'num_objects': 6124, 'num_object_clones': 0, 'num_object_copies': 12248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14066, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2449}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:57.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:19:57.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13654", 'reported_seq': 14760, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.911433+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:19:51.911433+0000', 'last_peered': '2024-09-11T05:19:51.911433+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:19:51.911433+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'3600", 'ondisk_log_start': "43'3600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24834072, 'num_objects': 6064, 'num_object_clones': 0, 'num_object_copies': 12128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6064, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12057, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 489, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3058}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13963", 'reported_seq': 14100, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.783915+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:19:51.783915+0000', 'last_peered': '2024-09-11T05:19:51.783915+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:19:51.783915+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25554944, 'num_objects': 6239, 'num_object_clones': 0, 'num_object_copies': 12478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11128, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13814", 'reported_seq': 13951, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.973041+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:19:51.973041+0000', 'last_peered': '2024-09-11T05:19:51.973041+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:19:51.973041+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25219072, 'num_objects': 6157, 'num_object_clones': 0, 'num_object_copies': 12314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11882, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2632}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13555", 'reported_seq': 13725, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.840990+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:19:51.840990+0000', 'last_peered': '2024-09-11T05:19:51.840990+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:19:51.840990+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'3500", 'ondisk_log_start': "43'3500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25055232, 'num_objects': 6117, 'num_object_clones': 0, 'num_object_copies': 12234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10308, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:19:57.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13834", 'reported_seq': 13957, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.939274+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:19:51.939274+0000', 'last_peered': '2024-09-11T05:19:51.939274+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:19:51.939274+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25391104, 'num_objects': 6199, 'num_object_clones': 0, 'num_object_copies': 12398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11863, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2677}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14004", 'reported_seq': 14140, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.892456+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:19:51.892456+0000', 'last_peered': '2024-09-11T05:19:51.892456+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:19:51.892456+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25149440, 'num_objects': 6140, 'num_object_clones': 0, 'num_object_copies': 12280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6140, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11037, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2517}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13603", 'reported_seq': 13742, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.752480+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:19:51.752480+0000', 'last_peered': '2024-09-11T05:19:51.752480+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:19:51.752480+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'3600", 'ondisk_log_start': "43'3600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24784896, 'num_objects': 6051, 'num_object_clones': 0, 'num_object_copies': 12102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10469, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13738", 'reported_seq': 13878, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.888601+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:19:51.888601+0000', 'last_peered': '2024-09-11T05:19:51.888601+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:19:51.888601+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'3700", 'ondisk_log_start': "43'3700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24829952, 'num_objects': 6062, 'num_object_clones': 0, 'num_object_copies': 12124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10496, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2518}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:19:57.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13606", 'reported_seq': 13755, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:19:51.878867+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:19:51.878867+0000', 'last_peered': '2024-09-11T05:19:51.878867+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:19:51.878867+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'3600", 'ondisk_log_start': "43'3600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24511081, 'num_objects': 5985, 'num_object_clones': 0, 'num_object_copies': 11970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10613, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:19:57.489 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:19:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:58.465 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34854/191128 objects degraded (18.236%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":476,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":95564,"data_bytes":391414420,"bytes_used":1966653440,"bytes_avail":287943639040,"bytes_total":289910292480,"degraded_objects":34854,"degraded_total":191128,"degraded_ratio":0.18235946590766397,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":554},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:19:58.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:19:58.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:19:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:19:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:19:59.422 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-11T05:19:57.561910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":391414420,"num_objects":95564,"num_object_clones":0,"num_object_copies":191128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":186135,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7659,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1920560,"kb_used_data":941172,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281194960,"statfs":{"total":289910292480,"available":287943639040,"internally_reserved":0,"allocated":963760128,"data_stored":962888790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1002818561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,6,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27254784,"num_objects":-6654,"num_object_clones":0,"num_object_copies":-13308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6654,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007233"},"pg_stats":[{"pgid":"2.f","version":"47'14176","reported_seq":14260,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.830475+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:56.830475+0000","last_peered":"2024-09-11T05:19:56.830475+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:56.830475+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2339}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13926","reported_seq":14032,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:57.019990+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:57.019990+0000","last_peered":"2024-09-11T05:19:57.019990+0000","last_clean":"2024-09-11T05:19:57.019990+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:57.019990+0000","last_undegraded":"2024-09-11T05:19:57.019990+0000","last_fullsized":"2024-09-11T05:19:57.019990+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24469504,"num_objects":5974,"num_object_clones":0,"num_object_copies":11948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12324,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13814","reported_seq":15094,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:57.001428+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:57.001428+0000","last_peered":"2024-09-11T05:19:57.001428+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:57.001428+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178712,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12217,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14080","reported_seq":14156,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:57.000392+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:57.000392+0000","last_peered":"2024-09-11T05:19:57.000392+0000","last_clean":"2024-09-11T05:19:57.000392+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:57.000392+0000","last_undegraded":"2024-09-11T05:19:57.000392+0000","last_fullsized":"2024-09-11T05:19:57.000392+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24084480,"num_objects":5880,"num_object_clones":0,"num_object_copies":11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14150","reported_seq":14283,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:56.828105+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:56.828105+0000","last_peered":"2024-09-11T05:19:56.828105+0000","last_clean":"2024-09-11T05:19:56.828105+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:56.828105+0000","last_undegraded":"2024-09-11T05:19:56.828105+0000","last_fullsized":"2024-09-11T05:19:56.828105+0000","mapping_epoch":34,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24653824,"num_objects":6019,"num_object_clones":0,"num_object_copies":12038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10984,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13967","reported_seq":21318,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:54.107294+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:54.107294+0000","last_peered":"2024-09-11T05:19:54.107294+0000","last_clean":"2024-09-11T05:19:54.107294+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:54.107294+0000","last_undegraded":"2024-09-11T05:19:54.107294+0000","last_fullsized":"2024-09-11T05:19:54.107294+0000","mapping_epoch":46,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24854528,"num_objects":6068,"num_object_clones":0,"num_object_copies":12136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10511,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14110","reported_seq":14248,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.763437+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:56.763437+0000","last_peered":"2024-09-11T05:19:56.763437+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:56.763437+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11275,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13869","reported_seq":13998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:56.879853+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:56.879853+0000","last_peered":"2024-09-11T05:19:56.879853+0000","last_clean":"2024-09-11T05:19:56.879853+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:56.879853+0000","last_undegraded":"2024-09-11T05:19:56.879853+0000","last_fullsized":"2024-09-11T05:19:56.879853+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11172,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13967","reported_seq":14104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.980146+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:56.980146+0000","last_peered":"2024-09-11T05:19:56.980146+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:56.980146+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2479}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13702","reported_seq":13872,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.710311+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:56.710311+0000","last_peered":"2024-09-11T05:19:56.710311+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:56.710311+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24453120,"num_objects":5970,"num_object_clones":0,"num_object_copies":11940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10455,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13957","reported_seq":14080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.923987+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:56.923987+0000","last_peered":"2024-09-11T05:19:56.923987+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:56.923987+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11986,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13812","reported_seq":20903,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:56.971422+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:56.971422+0000","last_peered":"2024-09-11T05:19:56.971422+0000","last_clean":"2024-09-11T05:19:56.971422+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:56.971422+0000","last_undegraded":"2024-09-11T05:19:56.971422+0000","last_fullsized":"2024-09-11T05:19:56.971422+0000","mapping_epoch":44,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14168","reported_seq":14305,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.805575+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:56.805575+0000","last_peered":"2024-09-11T05:19:56.805575+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:56.805575+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24477696,"num_objects":5976,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11201,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13740","reported_seq":13879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.882844+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:56.882844+0000","last_peered":"2024-09-11T05:19:56.882844+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:56.882844+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24223744,"num_objects":5914,"num_object_clones":0,"num_object_copies":11828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13863","reported_seq":14002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.773012+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:56.773012+0000","last_peered":"2024-09-11T05:19:56.773012+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:56.773012+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24317952,"num_objects":5937,"num_object_clones":0,"num_object_copies":11874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10621,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2393}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13768","reported_seq":13915,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.850592+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:56.850592+0000","last_peered":"2024-09-11T05:19:56.850592+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:56.850592+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23847529,"num_objects":5823,"num_object_clones":0,"num_object_copies":11646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10775,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":391414401,"num_objects":95562,"num_object_clones":0,"num_object_copies":191124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":186133,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7658,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":960241664,"data_stored":960163940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586004,"kb_used_data":305784,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93785836,"statfs":{"total":96636764160,"available":96036696064,"internally_reserved":0,"allocated":313122816,"data_stored":312853445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620868,"kb_used_data":292084,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750972,"statfs":{"total":96636764160,"available":96000995328,"internally_reserved":0,"allocated":299094016,"data_stored":298761425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215193,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713688,"kb_used_data":343304,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93658152,"statfs":{"total":96636764160,"available":95905947648,"internally_reserved":0,"allocated":351543296,"data_stored":351273920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379252395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350773248,"data_stored":350769769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297246720,"data_stored":297180026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312221696,"data_stored":312214145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:02.424 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:03.255 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:03.256 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:03.306 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-11T05:20:01.562868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":390910612,"num_objects":95441,"num_object_clones":0,"num_object_copies":190882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95441,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":186258,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7659,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160795,"ondisk_log_size":160795,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1901220,"kb_used_data":934312,"kb_used_omap":12,"kb_used_meta":966835,"kb_avail":281214300,"statfs":{"total":289910292480,"available":287963443200,"internally_reserved":0,"allocated":956735488,"data_stored":955864150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":990039041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,5,12,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18550784,"num_objects":-4529,"num_object_clones":0,"num_object_copies":-9058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4529,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006649"},"pg_stats":[{"pgid":"2.f","version":"47'14176","reported_seq":14260,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.830475+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:19:56.830475+0000","last_peered":"2024-09-11T05:19:56.830475+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:19:56.830475+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14176,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2339}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13926","reported_seq":14032,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:57.019990+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:19:57.019990+0000","last_peered":"2024-09-11T05:19:57.019990+0000","last_clean":"2024-09-11T05:19:57.019990+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:19:57.019990+0000","last_undegraded":"2024-09-11T05:19:57.019990+0000","last_fullsized":"2024-09-11T05:19:57.019990+0000","mapping_epoch":35,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24469504,"num_objects":5974,"num_object_clones":0,"num_object_copies":11948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12324,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13814","reported_seq":15094,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:57.001428+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:19:57.001428+0000","last_peered":"2024-09-11T05:19:57.001428+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:19:57.001428+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178712,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12217,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14080","reported_seq":14156,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:57.000392+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:19:57.000392+0000","last_peered":"2024-09-11T05:19:57.000392+0000","last_clean":"2024-09-11T05:19:57.000392+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:19:57.000392+0000","last_undegraded":"2024-09-11T05:19:57.000392+0000","last_fullsized":"2024-09-11T05:19:57.000392+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24084480,"num_objects":5880,"num_object_clones":0,"num_object_copies":11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14150","reported_seq":14283,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:56.828105+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:19:56.828105+0000","last_peered":"2024-09-11T05:19:56.828105+0000","last_clean":"2024-09-11T05:19:56.828105+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:19:56.828105+0000","last_undegraded":"2024-09-11T05:19:56.828105+0000","last_fullsized":"2024-09-11T05:19:56.828105+0000","mapping_epoch":34,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24653824,"num_objects":6019,"num_object_clones":0,"num_object_copies":12038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10984,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14090","reported_seq":21441,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:59.228804+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:59.228804+0000","last_peered":"2024-09-11T05:19:59.228804+0000","last_clean":"2024-09-11T05:19:59.228804+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:59.228804+0000","last_undegraded":"2024-09-11T05:19:59.228804+0000","last_fullsized":"2024-09-11T05:19:59.228804+0000","mapping_epoch":46,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24350720,"num_objects":5945,"num_object_clones":0,"num_object_copies":11890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10634,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14110","reported_seq":14248,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.763437+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:19:56.763437+0000","last_peered":"2024-09-11T05:19:56.763437+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:19:56.763437+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11275,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13869","reported_seq":13998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:56.879853+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:19:56.879853+0000","last_peered":"2024-09-11T05:19:56.879853+0000","last_clean":"2024-09-11T05:19:56.879853+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:19:56.879853+0000","last_undegraded":"2024-09-11T05:19:56.879853+0000","last_fullsized":"2024-09-11T05:19:56.879853+0000","mapping_epoch":34,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24637440,"num_objects":6015,"num_object_clones":0,"num_object_copies":12030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11172,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13967","reported_seq":14104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.980146+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:19:56.980146+0000","last_peered":"2024-09-11T05:19:56.980146+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:19:56.980146+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2479}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13702","reported_seq":13872,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.710311+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:19:56.710311+0000","last_peered":"2024-09-11T05:19:56.710311+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:19:56.710311+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24453120,"num_objects":5970,"num_object_clones":0,"num_object_copies":11940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10455,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13957","reported_seq":14080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.923987+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:19:56.923987+0000","last_peered":"2024-09-11T05:19:56.923987+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:19:56.923987+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11986,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13812","reported_seq":20903,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:56.971422+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:19:56.971422+0000","last_peered":"2024-09-11T05:19:56.971422+0000","last_clean":"2024-09-11T05:19:56.971422+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:19:56.971422+0000","last_undegraded":"2024-09-11T05:19:56.971422+0000","last_fullsized":"2024-09-11T05:19:56.971422+0000","mapping_epoch":44,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14168","reported_seq":14305,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.805575+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:19:56.805575+0000","last_peered":"2024-09-11T05:19:56.805575+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:19:56.805575+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24477696,"num_objects":5976,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11201,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13740","reported_seq":13879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.882844+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:19:56.882844+0000","last_peered":"2024-09-11T05:19:56.882844+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:19:56.882844+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24223744,"num_objects":5914,"num_object_clones":0,"num_object_copies":11828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13863","reported_seq":14002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.773012+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:19:56.773012+0000","last_peered":"2024-09-11T05:19:56.773012+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:19:56.773012+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24317952,"num_objects":5937,"num_object_clones":0,"num_object_copies":11874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10621,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2393}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13768","reported_seq":13915,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:19:56.850592+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:19:56.850592+0000","last_peered":"2024-09-11T05:19:56.850592+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:19:56.850592+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'3700","ondisk_log_start":"43'3700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23847529,"num_objects":5823,"num_object_clones":0,"num_object_copies":11646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10775,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":390910593,"num_objects":95439,"num_object_clones":0,"num_object_copies":190878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":186256,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7658,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":953368576,"data_stored":953290852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160792,"ondisk_log_size":160792,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586004,"kb_used_data":305784,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93785836,"statfs":{"total":96636764160,"available":96036696064,"internally_reserved":0,"allocated":313122816,"data_stored":312853445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286912171},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601528,"kb_used_data":285224,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93770312,"statfs":{"total":96636764160,"available":96020799488,"internally_reserved":0,"allocated":292069376,"data_stored":291736785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215193,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713688,"kb_used_data":343304,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93658152,"statfs":{"total":96636764160,"available":95905947648,"internally_reserved":0,"allocated":351543296,"data_stored":351273920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379252395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350773248,"data_stored":350769769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290373632,"data_stored":290306938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312221696,"data_stored":312214145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:03.308 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:04.031 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:04.034 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:04.118 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-11T05:20:03.563294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":382186132,"num_objects":93311,"num_object_clones":0,"num_object_copies":186622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188388,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7748,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160825,"ondisk_log_size":160825,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898220,"kb_used_data":922996,"kb_used_omap":12,"kb_used_meta":975155,"kb_avail":281217300,"statfs":{"total":289910292480,"available":287966515200,"internally_reserved":0,"allocated":945147904,"data_stored":944276566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":998558721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,9,8,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27275264,"num_objects":-6659,"num_object_clones":0,"num_object_copies":-13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6659,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005977"},"pg_stats":[{"pgid":"2.f","version":"47'14331","reported_seq":14415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.859957+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:01.859957+0000","last_peered":"2024-09-11T05:20:01.859957+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:01.859957+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14331,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14082","reported_seq":14190,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:02.008656+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:02.008656+0000","last_peered":"2024-09-11T05:20:02.008656+0000","last_clean":"2024-09-11T05:20:02.008656+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:02.008656+0000","last_undegraded":"2024-09-11T05:20:02.008656+0000","last_fullsized":"2024-09-11T05:20:02.008656+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23830528,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12480,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13961","reported_seq":15419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.990470+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:01.990470+0000","last_peered":"2024-09-11T05:20:01.990470+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:01.990470+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23576600,"num_objects":5757,"num_object_clones":0,"num_object_copies":11514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12364,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":664,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2926}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14235","reported_seq":14311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.942679+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:01.942679+0000","last_peered":"2024-09-11T05:20:01.942679+0000","last_clean":"2024-09-11T05:20:01.942679+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:01.942679+0000","last_undegraded":"2024-09-11T05:20:01.942679+0000","last_fullsized":"2024-09-11T05:20:01.942679+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23449600,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14235,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14295","reported_seq":14430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.844294+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:01.844294+0000","last_peered":"2024-09-11T05:20:01.844294+0000","last_clean":"2024-09-11T05:20:01.844294+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:01.844294+0000","last_undegraded":"2024-09-11T05:20:01.844294+0000","last_fullsized":"2024-09-11T05:20:01.844294+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11129,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14090","reported_seq":21441,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:59.228804+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:19:59.228804+0000","last_peered":"2024-09-11T05:19:59.228804+0000","last_clean":"2024-09-11T05:19:59.228804+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:19:59.228804+0000","last_undegraded":"2024-09-11T05:19:59.228804+0000","last_fullsized":"2024-09-11T05:19:59.228804+0000","mapping_epoch":46,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24350720,"num_objects":5945,"num_object_clones":0,"num_object_copies":11890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10634,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14250","reported_seq":14388,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.822586+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:01.822586+0000","last_peered":"2024-09-11T05:20:01.822586+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:01.822586+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11415,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14015","reported_seq":14144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.891087+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:01.891087+0000","last_peered":"2024-09-11T05:20:01.891087+0000","last_clean":"2024-09-11T05:20:01.891087+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:01.891087+0000","last_undegraded":"2024-09-11T05:20:01.891087+0000","last_fullsized":"2024-09-11T05:20:01.891087+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24039424,"num_objects":5869,"num_object_clones":0,"num_object_copies":11738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11318,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14113","reported_seq":14250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:02.006852+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:02.006852+0000","last_peered":"2024-09-11T05:20:02.006852+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:02.006852+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994368,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12181,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2333}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13851","reported_seq":14021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.921883+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:01.921883+0000","last_peered":"2024-09-11T05:20:01.921883+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:01.921883+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10604,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14104","reported_seq":14226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.952274+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:01.952274+0000","last_peered":"2024-09-11T05:20:01.952274+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:01.952274+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12133,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2407}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13945","reported_seq":21036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.995460+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:01.995460+0000","last_peered":"2024-09-11T05:20:01.995460+0000","last_clean":"2024-09-11T05:20:01.995460+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:01.995460+0000","last_undegraded":"2024-09-11T05:20:01.995460+0000","last_fullsized":"2024-09-11T05:20:01.995460+0000","mapping_epoch":44,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11848,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14305","reported_seq":14441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.859969+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:01.859969+0000","last_peered":"2024-09-11T05:20:01.859969+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:01.859969+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23916544,"num_objects":5839,"num_object_clones":0,"num_object_copies":11678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11338,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13880","reported_seq":14017,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.931084+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:01.931084+0000","last_peered":"2024-09-11T05:20:01.931084+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:01.931084+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10746,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13972","reported_seq":14112,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.883375+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:01.883375+0000","last_peered":"2024-09-11T05:20:01.883375+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:01.883375+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10730,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13893","reported_seq":14042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.894097+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:01.894097+0000","last_peered":"2024-09-11T05:20:01.894097+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:01.894097+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335529,"num_objects":5698,"num_object_clones":0,"num_object_copies":11396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10900,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":382186113,"num_objects":93309,"num_object_clones":0,"num_object_copies":186618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188386,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7747,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":941608960,"data_stored":941531236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585664,"kb_used_data":301292,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93786176,"statfs":{"total":96636764160,"available":96037044224,"internally_reserved":0,"allocated":308523008,"data_stored":308253637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601528,"kb_used_data":285224,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93770312,"statfs":{"total":96636764160,"available":96020799488,"internally_reserved":0,"allocated":292069376,"data_stored":291736785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215194,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711028,"kb_used_data":336480,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93660812,"statfs":{"total":96636764160,"available":95908671488,"internally_reserved":0,"allocated":344555520,"data_stored":344286144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343826432,"data_stored":343822953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290373632,"data_stored":290306938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307408896,"data_stored":307401345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14331", 'reported_seq': 14415, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.859957+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:01.859957+0000', 'last_peered': '2024-09-11T05:20:01.859957+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:01.859957+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23998464, 'num_objects': 5859, 'num_object_clones': 0, 'num_object_copies': 11718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14331, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2184}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13961", 'reported_seq': 15419, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.990470+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:01.990470+0000', 'last_peered': '2024-09-11T05:20:01.990470+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:01.990470+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23576600, 'num_objects': 5757, 'num_object_clones': 0, 'num_object_copies': 11514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5757, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12364, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 664, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2926}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14250", 'reported_seq': 14388, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.822586+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:01.822586+0000', 'last_peered': '2024-09-11T05:20:01.822586+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:01.822586+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24379392, 'num_objects': 5952, 'num_object_clones': 0, 'num_object_copies': 11904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11415, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:04.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14113", 'reported_seq': 14250, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:02.006852+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:02.006852+0000', 'last_peered': '2024-09-11T05:20:02.006852+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:02.006852+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23994368, 'num_objects': 5858, 'num_object_clones': 0, 'num_object_copies': 11716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5858, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12181, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2333}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13851", 'reported_seq': 14021, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.921883+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:01.921883+0000', 'last_peered': '2024-09-11T05:20:01.921883+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:01.921883+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23842816, 'num_objects': 5821, 'num_object_clones': 0, 'num_object_copies': 11642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10604, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2379}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14104", 'reported_seq': 14226, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.952274+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:01.952274+0000', 'last_peered': '2024-09-11T05:20:01.952274+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:01.952274+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24285184, 'num_objects': 5929, 'num_object_clones': 0, 'num_object_copies': 11858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12133, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2407}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14305", 'reported_seq': 14441, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.859969+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:01.859969+0000', 'last_peered': '2024-09-11T05:20:01.859969+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:01.859969+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23916544, 'num_objects': 5839, 'num_object_clones': 0, 'num_object_copies': 11678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11338, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2216}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'13880", 'reported_seq': 14017, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.931084+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:01.931084+0000', 'last_peered': '2024-09-11T05:20:01.931084+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:01.931084+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23650304, 'num_objects': 5774, 'num_object_clones': 0, 'num_object_copies': 11548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5774, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10746, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:04.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13972", 'reported_seq': 14112, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.883375+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:01.883375+0000', 'last_peered': '2024-09-11T05:20:01.883375+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:01.883375+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23871488, 'num_objects': 5828, 'num_object_clones': 0, 'num_object_copies': 11656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10730, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:04.122 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:04.122 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13893", 'reported_seq': 14042, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:01.894097+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:01.894097+0000', 'last_peered': '2024-09-11T05:20:01.894097+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:01.894097+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'3800", 'ondisk_log_start': "43'3800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23335529, 'num_objects': 5698, 'num_object_clones': 0, 'num_object_copies': 11396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5698, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10900, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2183}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:04.122 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:05.250 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:05.250 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34765/186622 objects degraded (18.629%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":483,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":93311,"data_bytes":382186132,"bytes_used":1943777280,"bytes_avail":287966515200,"bytes_total":289910292480,"degraded_objects":34765,"degraded_total":186622,"degraded_ratio":0.18628564692265651,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":554},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:05.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:05.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:06.122 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:06.125 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:06.219 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-11T05:20:05.563795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":381588116,"num_objects":93165,"num_object_clones":0,"num_object_copies":186330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188534,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7748,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898564,"kb_used_data":915016,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281216956,"statfs":{"total":289910292480,"available":287966162944,"internally_reserved":0,"allocated":936976384,"data_stored":936105046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1007078401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,2,8,6,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19132416,"num_objects":-4671,"num_object_clones":0,"num_object_copies":-9342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004371"},"pg_stats":[{"pgid":"2.f","version":"47'14331","reported_seq":14415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.859957+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:01.859957+0000","last_peered":"2024-09-11T05:20:01.859957+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:01.859957+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14331,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14082","reported_seq":14190,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:02.008656+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:02.008656+0000","last_peered":"2024-09-11T05:20:02.008656+0000","last_clean":"2024-09-11T05:20:02.008656+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:02.008656+0000","last_undegraded":"2024-09-11T05:20:02.008656+0000","last_fullsized":"2024-09-11T05:20:02.008656+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23830528,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12480,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13961","reported_seq":15419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.990470+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:01.990470+0000","last_peered":"2024-09-11T05:20:01.990470+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:01.990470+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23576600,"num_objects":5757,"num_object_clones":0,"num_object_copies":11514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12364,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":664,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2926}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14235","reported_seq":14311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.942679+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:01.942679+0000","last_peered":"2024-09-11T05:20:01.942679+0000","last_clean":"2024-09-11T05:20:01.942679+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:01.942679+0000","last_undegraded":"2024-09-11T05:20:01.942679+0000","last_fullsized":"2024-09-11T05:20:01.942679+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23449600,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14235,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14295","reported_seq":14430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.844294+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:01.844294+0000","last_peered":"2024-09-11T05:20:01.844294+0000","last_clean":"2024-09-11T05:20:01.844294+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:01.844294+0000","last_undegraded":"2024-09-11T05:20:01.844294+0000","last_fullsized":"2024-09-11T05:20:01.844294+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11129,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14236","reported_seq":21587,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:04.173765+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:04.173765+0000","last_peered":"2024-09-11T05:20:04.173765+0000","last_clean":"2024-09-11T05:20:04.173765+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:04.173765+0000","last_undegraded":"2024-09-11T05:20:04.173765+0000","last_fullsized":"2024-09-11T05:20:04.173765+0000","mapping_epoch":46,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10780,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14250","reported_seq":14388,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.822586+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:01.822586+0000","last_peered":"2024-09-11T05:20:01.822586+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:01.822586+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11415,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14015","reported_seq":14144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.891087+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:01.891087+0000","last_peered":"2024-09-11T05:20:01.891087+0000","last_clean":"2024-09-11T05:20:01.891087+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:01.891087+0000","last_undegraded":"2024-09-11T05:20:01.891087+0000","last_fullsized":"2024-09-11T05:20:01.891087+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24039424,"num_objects":5869,"num_object_clones":0,"num_object_copies":11738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11318,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14113","reported_seq":14250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:02.006852+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:02.006852+0000","last_peered":"2024-09-11T05:20:02.006852+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:02.006852+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994368,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12181,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2333}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13851","reported_seq":14021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.921883+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:01.921883+0000","last_peered":"2024-09-11T05:20:01.921883+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:01.921883+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10604,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14104","reported_seq":14226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.952274+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:01.952274+0000","last_peered":"2024-09-11T05:20:01.952274+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:01.952274+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12133,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2407}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13945","reported_seq":21036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:01.995460+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:01.995460+0000","last_peered":"2024-09-11T05:20:01.995460+0000","last_clean":"2024-09-11T05:20:01.995460+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:01.995460+0000","last_undegraded":"2024-09-11T05:20:01.995460+0000","last_fullsized":"2024-09-11T05:20:01.995460+0000","mapping_epoch":44,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11848,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14305","reported_seq":14441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.859969+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:01.859969+0000","last_peered":"2024-09-11T05:20:01.859969+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:01.859969+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23916544,"num_objects":5839,"num_object_clones":0,"num_object_copies":11678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11338,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13880","reported_seq":14017,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.931084+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:01.931084+0000","last_peered":"2024-09-11T05:20:01.931084+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:01.931084+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10746,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13972","reported_seq":14112,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.883375+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:01.883375+0000","last_peered":"2024-09-11T05:20:01.883375+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:01.883375+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10730,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13893","reported_seq":14042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:01.894097+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:01.894097+0000","last_peered":"2024-09-11T05:20:01.894097+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:01.894097+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'3800","ondisk_log_start":"43'3800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23335529,"num_objects":5698,"num_object_clones":0,"num_object_copies":11396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10900,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":381588097,"num_objects":93163,"num_object_clones":0,"num_object_copies":186326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188532,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7747,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":934633472,"data_stored":934555748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585664,"kb_used_data":301292,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93786176,"statfs":{"total":96636764160,"available":96037044224,"internally_reserved":0,"allocated":308523008,"data_stored":308253637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291172011},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601872,"kb_used_data":277244,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93769968,"statfs":{"total":96636764160,"available":96020447232,"internally_reserved":0,"allocated":283897856,"data_stored":283565265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215194,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711028,"kb_used_data":336480,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93660812,"statfs":{"total":96636764160,"available":95908671488,"internally_reserved":0,"allocated":344555520,"data_stored":344286144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343826432,"data_stored":343822953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283398144,"data_stored":283331450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307408896,"data_stored":307401345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:09.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:10.053 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:10.054 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:10.103 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-11T05:20:09.565390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372503188,"num_objects":90947,"num_object_clones":0,"num_object_copies":181894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190752,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7838,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1885040,"kb_used_data":897336,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281230480,"statfs":{"total":289910292480,"available":287980011520,"internally_reserved":0,"allocated":918872064,"data_stored":918000726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1011338241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,6,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18911232,"num_objects":-4617,"num_object_clones":0,"num_object_copies":-9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003480"},"pg_stats":[{"pgid":"2.f","version":"47'14485","reported_seq":14569,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.949294+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:06.949294+0000","last_peered":"2024-09-11T05:20:06.949294+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:06.949294+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23367680,"num_objects":5705,"num_object_clones":0,"num_object_copies":11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14485,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14230","reported_seq":14338,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.001387+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:07.001387+0000","last_peered":"2024-09-11T05:20:07.001387+0000","last_clean":"2024-09-11T05:20:07.001387+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:07.001387+0000","last_undegraded":"2024-09-11T05:20:07.001387+0000","last_fullsized":"2024-09-11T05:20:07.001387+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12628,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14104","reported_seq":15744,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.987774+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:06.987774+0000","last_peered":"2024-09-11T05:20:06.987774+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:06.987774+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990872,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12507,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2873}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14375","reported_seq":14452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.012585+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:07.012585+0000","last_peered":"2024-09-11T05:20:07.012585+0000","last_clean":"2024-09-11T05:20:07.012585+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:07.012585+0000","last_undegraded":"2024-09-11T05:20:07.012585+0000","last_fullsized":"2024-09-11T05:20:07.012585+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22876160,"num_objects":5585,"num_object_clones":0,"num_object_copies":11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14375,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14415","reported_seq":14550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:06.931555+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:06.931555+0000","last_peered":"2024-09-11T05:20:06.931555+0000","last_clean":"2024-09-11T05:20:06.931555+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:06.931555+0000","last_undegraded":"2024-09-11T05:20:06.931555+0000","last_fullsized":"2024-09-11T05:20:06.931555+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11249,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14400","reported_seq":21751,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:09.302127+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:09.302127+0000","last_peered":"2024-09-11T05:20:09.302127+0000","last_clean":"2024-09-11T05:20:09.302127+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:09.302127+0000","last_undegraded":"2024-09-11T05:20:09.302127+0000","last_fullsized":"2024-09-11T05:20:09.302127+0000","mapping_epoch":46,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14397","reported_seq":14535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.917460+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:06.917460+0000","last_peered":"2024-09-11T05:20:06.917460+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:06.917460+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11562,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14149","reported_seq":14278,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:06.889404+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:06.889404+0000","last_peered":"2024-09-11T05:20:06.889404+0000","last_clean":"2024-09-11T05:20:06.889404+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:06.889404+0000","last_undegraded":"2024-09-11T05:20:06.889404+0000","last_fullsized":"2024-09-11T05:20:06.889404+0000","mapping_epoch":34,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11452,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14251","reported_seq":14388,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.985063+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:06.985063+0000","last_peered":"2024-09-11T05:20:06.985063+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:06.985063+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23429120,"num_objects":5720,"num_object_clones":0,"num_object_copies":11440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2195}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13986","reported_seq":14156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.882477+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:06.882477+0000","last_peered":"2024-09-11T05:20:06.882477+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:06.882477+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23289856,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10739,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2244}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14238","reported_seq":14361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.965219+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:06.965219+0000","last_peered":"2024-09-11T05:20:06.965219+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:06.965219+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12267,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14071","reported_seq":21162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.030093+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:07.030093+0000","last_peered":"2024-09-11T05:20:07.030093+0000","last_clean":"2024-09-11T05:20:07.030093+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:07.030093+0000","last_undegraded":"2024-09-11T05:20:07.030093+0000","last_fullsized":"2024-09-11T05:20:07.030093+0000","mapping_epoch":44,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11974,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14442","reported_seq":14578,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.970559+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:06.970559+0000","last_peered":"2024-09-11T05:20:06.970559+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:06.970559+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11475,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2079}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14008","reported_seq":14147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.908501+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:06.908501+0000","last_peered":"2024-09-11T05:20:06.908501+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:06.908501+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23126016,"num_objects":5646,"num_object_clones":0,"num_object_copies":11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10874,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14095","reported_seq":14233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.941800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:06.941800+0000","last_peered":"2024-09-11T05:20:06.941800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:06.941800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23367680,"num_objects":5705,"num_object_clones":0,"num_object_copies":11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10853,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14040","reported_seq":14189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.946148+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:06.946148+0000","last_peered":"2024-09-11T05:20:06.946148+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:06.946148+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22733417,"num_objects":5551,"num_object_clones":0,"num_object_copies":11102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":372503169,"num_objects":90945,"num_object_clones":0,"num_object_copies":181890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90945,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190750,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7837,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":916373504,"data_stored":916295780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585528,"kb_used_data":296992,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93786312,"statfs":{"total":96636764160,"available":96037183488,"internally_reserved":0,"allocated":304119808,"data_stored":303850437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586684,"kb_used_data":270380,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93785156,"statfs":{"total":96636764160,"available":96035999744,"internally_reserved":0,"allocated":276869120,"data_stored":276536529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,3,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215195,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712828,"kb_used_data":329964,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93659012,"statfs":{"total":96636764160,"available":95906828288,"internally_reserved":0,"allocated":337883136,"data_stored":337613760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":337018880,"data_stored":337015401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276393984,"data_stored":276327290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302960640,"data_stored":302953089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:10.104 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:10.949 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:10.950 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:11.026 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-11T05:20:09.565390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372503188,"num_objects":90947,"num_object_clones":0,"num_object_copies":181894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190752,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7838,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1885040,"kb_used_data":897336,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281230480,"statfs":{"total":289910292480,"available":287980011520,"internally_reserved":0,"allocated":918872064,"data_stored":918000726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1011338241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,6,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18911232,"num_objects":-4617,"num_object_clones":0,"num_object_copies":-9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003480"},"pg_stats":[{"pgid":"2.f","version":"47'14485","reported_seq":14569,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.949294+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:06.949294+0000","last_peered":"2024-09-11T05:20:06.949294+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:06.949294+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23367680,"num_objects":5705,"num_object_clones":0,"num_object_copies":11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14485,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14230","reported_seq":14338,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.001387+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:07.001387+0000","last_peered":"2024-09-11T05:20:07.001387+0000","last_clean":"2024-09-11T05:20:07.001387+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:07.001387+0000","last_undegraded":"2024-09-11T05:20:07.001387+0000","last_fullsized":"2024-09-11T05:20:07.001387+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12628,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14104","reported_seq":15744,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.987774+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:06.987774+0000","last_peered":"2024-09-11T05:20:06.987774+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:06.987774+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990872,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12507,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2873}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14375","reported_seq":14452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.012585+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:07.012585+0000","last_peered":"2024-09-11T05:20:07.012585+0000","last_clean":"2024-09-11T05:20:07.012585+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:07.012585+0000","last_undegraded":"2024-09-11T05:20:07.012585+0000","last_fullsized":"2024-09-11T05:20:07.012585+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22876160,"num_objects":5585,"num_object_clones":0,"num_object_copies":11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14375,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14415","reported_seq":14550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:06.931555+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:06.931555+0000","last_peered":"2024-09-11T05:20:06.931555+0000","last_clean":"2024-09-11T05:20:06.931555+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:06.931555+0000","last_undegraded":"2024-09-11T05:20:06.931555+0000","last_fullsized":"2024-09-11T05:20:06.931555+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11249,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14400","reported_seq":21751,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:09.302127+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:09.302127+0000","last_peered":"2024-09-11T05:20:09.302127+0000","last_clean":"2024-09-11T05:20:09.302127+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:09.302127+0000","last_undegraded":"2024-09-11T05:20:09.302127+0000","last_fullsized":"2024-09-11T05:20:09.302127+0000","mapping_epoch":46,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14397","reported_seq":14535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.917460+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:06.917460+0000","last_peered":"2024-09-11T05:20:06.917460+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:06.917460+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11562,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14149","reported_seq":14278,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:06.889404+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:06.889404+0000","last_peered":"2024-09-11T05:20:06.889404+0000","last_clean":"2024-09-11T05:20:06.889404+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:06.889404+0000","last_undegraded":"2024-09-11T05:20:06.889404+0000","last_fullsized":"2024-09-11T05:20:06.889404+0000","mapping_epoch":34,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11452,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14251","reported_seq":14388,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.985063+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:06.985063+0000","last_peered":"2024-09-11T05:20:06.985063+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:06.985063+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23429120,"num_objects":5720,"num_object_clones":0,"num_object_copies":11440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2195}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13986","reported_seq":14156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.882477+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:06.882477+0000","last_peered":"2024-09-11T05:20:06.882477+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:06.882477+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23289856,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10739,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2244}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14238","reported_seq":14361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.965219+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:06.965219+0000","last_peered":"2024-09-11T05:20:06.965219+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:06.965219+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12267,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14071","reported_seq":21162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.030093+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:07.030093+0000","last_peered":"2024-09-11T05:20:07.030093+0000","last_clean":"2024-09-11T05:20:07.030093+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:07.030093+0000","last_undegraded":"2024-09-11T05:20:07.030093+0000","last_fullsized":"2024-09-11T05:20:07.030093+0000","mapping_epoch":44,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11974,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14442","reported_seq":14578,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.970559+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:06.970559+0000","last_peered":"2024-09-11T05:20:06.970559+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:06.970559+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11475,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2079}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14008","reported_seq":14147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.908501+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:06.908501+0000","last_peered":"2024-09-11T05:20:06.908501+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:06.908501+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23126016,"num_objects":5646,"num_object_clones":0,"num_object_copies":11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10874,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14095","reported_seq":14233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.941800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:06.941800+0000","last_peered":"2024-09-11T05:20:06.941800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:06.941800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23367680,"num_objects":5705,"num_object_clones":0,"num_object_copies":11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10853,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14040","reported_seq":14189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.946148+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:06.946148+0000","last_peered":"2024-09-11T05:20:06.946148+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:06.946148+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22733417,"num_objects":5551,"num_object_clones":0,"num_object_copies":11102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":372503169,"num_objects":90945,"num_object_clones":0,"num_object_copies":181890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90945,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190750,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7837,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":916373504,"data_stored":916295780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585528,"kb_used_data":296992,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93786312,"statfs":{"total":96636764160,"available":96037183488,"internally_reserved":0,"allocated":304119808,"data_stored":303850437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586684,"kb_used_data":270380,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93785156,"statfs":{"total":96636764160,"available":96035999744,"internally_reserved":0,"allocated":276869120,"data_stored":276536529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,3,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215195,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712828,"kb_used_data":329964,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93659012,"statfs":{"total":96636764160,"available":95906828288,"internally_reserved":0,"allocated":337883136,"data_stored":337613760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":337018880,"data_stored":337015401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276393984,"data_stored":276327290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302960640,"data_stored":302953089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:11.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:11.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14485", 'reported_seq': 14569, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.949294+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:06.949294+0000', 'last_peered': '2024-09-11T05:20:06.949294+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:06.949294+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23367680, 'num_objects': 5705, 'num_object_clones': 0, 'num_object_copies': 11410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14485, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:11.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:11.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14104", 'reported_seq': 15744, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.987774+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:06.987774+0000', 'last_peered': '2024-09-11T05:20:06.987774+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:06.987774+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22990872, 'num_objects': 5614, 'num_object_clones': 0, 'num_object_copies': 11228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2741, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5614, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12507, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 754, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2873}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:11.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:11.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14397", 'reported_seq': 14535, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.917460+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:06.917460+0000', 'last_peered': '2024-09-11T05:20:06.917460+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:06.917460+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23777280, 'num_objects': 5805, 'num_object_clones': 0, 'num_object_copies': 11610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11562, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2229}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14251", 'reported_seq': 14388, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.985063+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:06.985063+0000', 'last_peered': '2024-09-11T05:20:06.985063+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:06.985063+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23429120, 'num_objects': 5720, 'num_object_clones': 0, 'num_object_copies': 11440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12319, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2195}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13986", 'reported_seq': 14156, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.882477+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:06.882477+0000', 'last_peered': '2024-09-11T05:20:06.882477+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:06.882477+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'3900", 'ondisk_log_start': "43'3900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23289856, 'num_objects': 5686, 'num_object_clones': 0, 'num_object_copies': 11372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10739, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2244}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14238", 'reported_seq': 14361, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.965219+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:06.965219+0000', 'last_peered': '2024-09-11T05:20:06.965219+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:06.965219+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23736320, 'num_objects': 5795, 'num_object_clones': 0, 'num_object_copies': 11590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5795, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12267, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2273}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14442", 'reported_seq': 14578, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.970559+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:06.970559+0000', 'last_peered': '2024-09-11T05:20:06.970559+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:06.970559+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23355392, 'num_objects': 5702, 'num_object_clones': 0, 'num_object_copies': 11404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11475, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2079}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:11.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14008", 'reported_seq': 14147, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.908501+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:06.908501+0000', 'last_peered': '2024-09-11T05:20:06.908501+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:06.908501+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23126016, 'num_objects': 5646, 'num_object_clones': 0, 'num_object_copies': 11292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10874, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2151}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:11.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:11.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14095", 'reported_seq': 14233, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.941800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:06.941800+0000', 'last_peered': '2024-09-11T05:20:06.941800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:06.941800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23367680, 'num_objects': 5705, 'num_object_clones': 0, 'num_object_copies': 11410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10853, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:11.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:11.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14040", 'reported_seq': 14189, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:06.946148+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:06.946148+0000', 'last_peered': '2024-09-11T05:20:06.946148+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:06.946148+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22733417, 'num_objects': 5551, 'num_object_clones': 0, 'num_object_copies': 11102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11047, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2036}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:11.030 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:11.906 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:11.906 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34675/181894 objects degraded (19.063%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":489,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":90947,"data_bytes":372503188,"bytes_used":1930280960,"bytes_avail":287980011520,"bytes_total":289910292480,"degraded_objects":34675,"degraded_total":181894,"degraded_ratio":0.19063300603648281,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":179,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":374},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:11.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:11.963 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:12.769 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:12.770 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:12.834 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-11T05:20:11.565867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372503188,"num_objects":90947,"num_object_clones":0,"num_object_copies":181894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190752,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7838,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1885040,"kb_used_data":897336,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281230480,"statfs":{"total":289910292480,"available":287980011520,"internally_reserved":0,"allocated":918872064,"data_stored":918000726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1011338241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,6,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18407424,"num_objects":-4494,"num_object_clones":0,"num_object_copies":-8988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4494,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003471"},"pg_stats":[{"pgid":"2.f","version":"47'14485","reported_seq":14569,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.949294+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:06.949294+0000","last_peered":"2024-09-11T05:20:06.949294+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:06.949294+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23367680,"num_objects":5705,"num_object_clones":0,"num_object_copies":11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14485,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14230","reported_seq":14338,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.001387+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:07.001387+0000","last_peered":"2024-09-11T05:20:07.001387+0000","last_clean":"2024-09-11T05:20:07.001387+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:07.001387+0000","last_undegraded":"2024-09-11T05:20:07.001387+0000","last_fullsized":"2024-09-11T05:20:07.001387+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12628,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14104","reported_seq":15744,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.987774+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:06.987774+0000","last_peered":"2024-09-11T05:20:06.987774+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:06.987774+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990872,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12507,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2873}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14375","reported_seq":14452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.012585+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:07.012585+0000","last_peered":"2024-09-11T05:20:07.012585+0000","last_clean":"2024-09-11T05:20:07.012585+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:07.012585+0000","last_undegraded":"2024-09-11T05:20:07.012585+0000","last_fullsized":"2024-09-11T05:20:07.012585+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22876160,"num_objects":5585,"num_object_clones":0,"num_object_copies":11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14375,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14415","reported_seq":14550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:06.931555+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:06.931555+0000","last_peered":"2024-09-11T05:20:06.931555+0000","last_clean":"2024-09-11T05:20:06.931555+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:06.931555+0000","last_undegraded":"2024-09-11T05:20:06.931555+0000","last_fullsized":"2024-09-11T05:20:06.931555+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23568384,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11249,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14400","reported_seq":21751,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:09.302127+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:09.302127+0000","last_peered":"2024-09-11T05:20:09.302127+0000","last_clean":"2024-09-11T05:20:09.302127+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:09.302127+0000","last_undegraded":"2024-09-11T05:20:09.302127+0000","last_fullsized":"2024-09-11T05:20:09.302127+0000","mapping_epoch":46,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14397","reported_seq":14535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.917460+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:06.917460+0000","last_peered":"2024-09-11T05:20:06.917460+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:06.917460+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23777280,"num_objects":5805,"num_object_clones":0,"num_object_copies":11610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11562,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14149","reported_seq":14278,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:06.889404+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:06.889404+0000","last_peered":"2024-09-11T05:20:06.889404+0000","last_clean":"2024-09-11T05:20:06.889404+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:06.889404+0000","last_undegraded":"2024-09-11T05:20:06.889404+0000","last_fullsized":"2024-09-11T05:20:06.889404+0000","mapping_epoch":34,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11452,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707510+0000","last_peered":"2024-09-11T05:19:25.707510+0000","last_clean":"2024-09-11T05:19:25.707510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707510+0000","last_undegraded":"2024-09-11T05:19:25.707510+0000","last_fullsized":"2024-09-11T05:19:25.707510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14251","reported_seq":14388,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.985063+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:06.985063+0000","last_peered":"2024-09-11T05:20:06.985063+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:06.985063+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23429120,"num_objects":5720,"num_object_clones":0,"num_object_copies":11440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2195}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13986","reported_seq":14156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.882477+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:06.882477+0000","last_peered":"2024-09-11T05:20:06.882477+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:06.882477+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'3900","ondisk_log_start":"43'3900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23289856,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10739,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2244}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14238","reported_seq":14361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.965219+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:06.965219+0000","last_peered":"2024-09-11T05:20:06.965219+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:06.965219+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12267,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14071","reported_seq":21162,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:07.030093+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:07.030093+0000","last_peered":"2024-09-11T05:20:07.030093+0000","last_clean":"2024-09-11T05:20:07.030093+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:07.030093+0000","last_undegraded":"2024-09-11T05:20:07.030093+0000","last_fullsized":"2024-09-11T05:20:07.030093+0000","mapping_epoch":44,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11974,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14442","reported_seq":14578,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.970559+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:06.970559+0000","last_peered":"2024-09-11T05:20:06.970559+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:06.970559+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11475,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2079}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14008","reported_seq":14147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.908501+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:06.908501+0000","last_peered":"2024-09-11T05:20:06.908501+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:06.908501+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23126016,"num_objects":5646,"num_object_clones":0,"num_object_copies":11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10874,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14095","reported_seq":14233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.941800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:06.941800+0000","last_peered":"2024-09-11T05:20:06.941800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:06.941800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23367680,"num_objects":5705,"num_object_clones":0,"num_object_copies":11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10853,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14040","reported_seq":14189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:06.946148+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:06.946148+0000","last_peered":"2024-09-11T05:20:06.946148+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:06.946148+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22733417,"num_objects":5551,"num_object_clones":0,"num_object_copies":11102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":372503169,"num_objects":90945,"num_object_clones":0,"num_object_copies":181890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90945,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190750,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7837,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":916373504,"data_stored":916295780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585528,"kb_used_data":296992,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93786312,"statfs":{"total":96636764160,"available":96037183488,"internally_reserved":0,"allocated":304119808,"data_stored":303850437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295431851},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586684,"kb_used_data":270380,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93785156,"statfs":{"total":96636764160,"available":96035999744,"internally_reserved":0,"allocated":276869120,"data_stored":276536529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,3,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215195,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712828,"kb_used_data":329964,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93659012,"statfs":{"total":96636764160,"available":95906828288,"internally_reserved":0,"allocated":337883136,"data_stored":337613760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":337018880,"data_stored":337015401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276393984,"data_stored":276327290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302960640,"data_stored":302953089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:15.836 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:16.747 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:16.750 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:16.791 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-11T05:20:15.566702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363250324,"num_objects":88688,"num_object_clones":0,"num_object_copies":177376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193011,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7935,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1880656,"kb_used_data":879064,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281234864,"statfs":{"total":289910292480,"available":287984500736,"internally_reserved":0,"allocated":900161536,"data_stored":899290198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1025559553},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,3,13,14,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18935808,"num_objects":-4623,"num_object_clones":0,"num_object_copies":-9246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4623,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003407"},"pg_stats":[{"pgid":"2.f","version":"47'14622","reported_seq":14705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:11.955561+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:11.955561+0000","last_peered":"2024-09-11T05:20:11.955561+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:11.955561+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":5568,"num_object_clones":0,"num_object_copies":11136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14622,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1893}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14377","reported_seq":14485,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:12.065618+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:12.065618+0000","last_peered":"2024-09-11T05:20:12.065618+0000","last_clean":"2024-09-11T05:20:12.065618+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:12.065618+0000","last_undegraded":"2024-09-11T05:20:12.065618+0000","last_fullsized":"2024-09-11T05:20:12.065618+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22622208,"num_objects":5523,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12775,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14228","reported_seq":16062,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:12.052436+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:12.052436+0000","last_peered":"2024-09-11T05:20:12.052436+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:12.052436+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22482968,"num_objects":5490,"num_object_clones":0,"num_object_copies":10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12631,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":851,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14518","reported_seq":14595,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:12.021291+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:12.021291+0000","last_peered":"2024-09-11T05:20:12.021291+0000","last_clean":"2024-09-11T05:20:12.021291+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:12.021291+0000","last_undegraded":"2024-09-11T05:20:12.021291+0000","last_fullsized":"2024-09-11T05:20:12.021291+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22290432,"num_objects":5442,"num_object_clones":0,"num_object_copies":10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14518,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14566","reported_seq":14701,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:11.976758+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:11.976758+0000","last_peered":"2024-09-11T05:20:11.976758+0000","last_clean":"2024-09-11T05:20:11.976758+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:11.976758+0000","last_undegraded":"2024-09-11T05:20:11.976758+0000","last_fullsized":"2024-09-11T05:20:11.976758+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22949888,"num_objects":5603,"num_object_clones":0,"num_object_copies":11206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11400,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14550","reported_seq":21901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:14.311007+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:14.311007+0000","last_peered":"2024-09-11T05:20:14.311007+0000","last_clean":"2024-09-11T05:20:14.311007+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:14.311007+0000","last_undegraded":"2024-09-11T05:20:14.311007+0000","last_fullsized":"2024-09-11T05:20:14.311007+0000","mapping_epoch":46,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11094,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14527","reported_seq":14665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:11.844489+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:11.844489+0000","last_peered":"2024-09-11T05:20:11.844489+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:11.844489+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23244800,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14288","reported_seq":14417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:11.971249+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:11.971249+0000","last_peered":"2024-09-11T05:20:11.971249+0000","last_clean":"2024-09-11T05:20:11.971249+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:11.971249+0000","last_undegraded":"2024-09-11T05:20:11.971249+0000","last_fullsized":"2024-09-11T05:20:11.971249+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22921216,"num_objects":5596,"num_object_clones":0,"num_object_copies":11192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11591,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14401","reported_seq":14538,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:12.038683+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:12.038683+0000","last_peered":"2024-09-11T05:20:12.038683+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:12.038683+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12469,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2045}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14129","reported_seq":14299,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:11.923735+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:11.923735+0000","last_peered":"2024-09-11T05:20:11.923735+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:11.923735+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10882,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14378","reported_seq":14501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:12.045655+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:12.045655+0000","last_peered":"2024-09-11T05:20:12.045655+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:12.045655+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23162880,"num_objects":5655,"num_object_clones":0,"num_object_copies":11310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12407,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2133}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14211","reported_seq":21302,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:12.036528+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:12.036528+0000","last_peered":"2024-09-11T05:20:12.036528+0000","last_clean":"2024-09-11T05:20:12.036528+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:12.036528+0000","last_undegraded":"2024-09-11T05:20:12.036528+0000","last_fullsized":"2024-09-11T05:20:12.036528+0000","mapping_epoch":44,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22515712,"num_objects":5497,"num_object_clones":0,"num_object_copies":10994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12114,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14584","reported_seq":14721,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:11.956288+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:11.956288+0000","last_peered":"2024-09-11T05:20:11.956288+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:11.956288+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22773760,"num_objects":5560,"num_object_clones":0,"num_object_copies":11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11617,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14151","reported_seq":14290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:11.924404+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:11.924404+0000","last_peered":"2024-09-11T05:20:11.924404+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:11.924404+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540288,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11017,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14230","reported_seq":14370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:11.896285+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:11.896285+0000","last_peered":"2024-09-11T05:20:11.896285+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:11.896285+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10988,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14185","reported_seq":14333,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:11.973804+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:11.973804+0000","last_peered":"2024-09-11T05:20:11.973804+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:11.973804+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22139497,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11192,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363250305,"num_objects":88686,"num_object_clones":0,"num_object_copies":177372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193009,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7934,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":897781760,"data_stored":897704036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603292,"kb_used_data":292548,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93768548,"statfs":{"total":96636764160,"available":96018993152,"internally_reserved":0,"allocated":299569152,"data_stored":299299781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584000,"kb_used_data":263536,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93787840,"statfs":{"total":96636764160,"available":96038748160,"internally_reserved":0,"allocated":269860864,"data_stored":269528273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215196,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693364,"kb_used_data":322980,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93678476,"statfs":{"total":96636764160,"available":95926759424,"internally_reserved":0,"allocated":330731520,"data_stored":330462144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379252395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,1,3,6,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":330002432,"data_stored":329998953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269443072,"data_stored":269376378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298336256,"data_stored":298328705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:16.793 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:17.624 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:17.625 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:17.668 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-11T05:20:17.569832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":354411156,"num_objects":86530,"num_object_clones":0,"num_object_copies":173060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86530,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195169,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8033,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1882936,"kb_used_data":867460,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281232584,"statfs":{"total":289910292480,"available":287982166016,"internally_reserved":0,"allocated":888279040,"data_stored":887407702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1039780865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,12,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27176960,"num_objects":-6635,"num_object_clones":0,"num_object_copies":-13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006037"},"pg_stats":[{"pgid":"2.f","version":"47'14762","reported_seq":14846,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.052002+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:17.052002+0000","last_peered":"2024-09-11T05:20:17.052002+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:17.052002+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22233088,"num_objects":5428,"num_object_clones":0,"num_object_copies":10856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1753}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14530","reported_seq":14638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.077883+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:17.077883+0000","last_peered":"2024-09-11T05:20:17.077883+0000","last_clean":"2024-09-11T05:20:17.077883+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:17.077883+0000","last_undegraded":"2024-09-11T05:20:17.077883+0000","last_fullsized":"2024-09-11T05:20:17.077883+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14359","reported_seq":16389,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.066681+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:17.066681+0000","last_peered":"2024-09-11T05:20:17.066681+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:17.066681+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21946392,"num_objects":5359,"num_object_clones":0,"num_object_copies":10718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12762,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":949,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2813}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14632","reported_seq":14709,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:16.984448+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:16.984448+0000","last_peered":"2024-09-11T05:20:16.984448+0000","last_clean":"2024-09-11T05:20:16.984448+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:16.984448+0000","last_undegraded":"2024-09-11T05:20:16.984448+0000","last_fullsized":"2024-09-11T05:20:16.984448+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14632,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14704","reported_seq":14839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.004750+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:17.004750+0000","last_peered":"2024-09-11T05:20:17.004750+0000","last_clean":"2024-09-11T05:20:17.004750+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:17.004750+0000","last_undegraded":"2024-09-11T05:20:17.004750+0000","last_fullsized":"2024-09-11T05:20:17.004750+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22384640,"num_objects":5465,"num_object_clones":0,"num_object_copies":10930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11538,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14550","reported_seq":21901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:14.311007+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:14.311007+0000","last_peered":"2024-09-11T05:20:14.311007+0000","last_clean":"2024-09-11T05:20:14.311007+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:14.311007+0000","last_undegraded":"2024-09-11T05:20:14.311007+0000","last_fullsized":"2024-09-11T05:20:14.311007+0000","mapping_epoch":46,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11094,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14669","reported_seq":14807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.996978+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:16.996978+0000","last_peered":"2024-09-11T05:20:16.996978+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:16.996978+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11834,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14447","reported_seq":14576,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.009569+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:17.009569+0000","last_peered":"2024-09-11T05:20:17.009569+0000","last_clean":"2024-09-11T05:20:17.009569+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:17.009569+0000","last_undegraded":"2024-09-11T05:20:17.009569+0000","last_fullsized":"2024-09-11T05:20:17.009569+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11750,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14551","reported_seq":14688,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.032432+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:17.032432+0000","last_peered":"2024-09-11T05:20:17.032432+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:17.032432+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12619,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1895}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14265","reported_seq":14435,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.982194+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:16.982194+0000","last_peered":"2024-09-11T05:20:16.982194+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:16.982194+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11018,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1965}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14526","reported_seq":14649,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.087341+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:17.087341+0000","last_peered":"2024-09-11T05:20:17.087341+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:17.087341+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12555,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1985}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14364","reported_seq":21455,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.014484+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:17.014484+0000","last_peered":"2024-09-11T05:20:17.014484+0000","last_clean":"2024-09-11T05:20:17.014484+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:17.014484+0000","last_undegraded":"2024-09-11T05:20:17.014484+0000","last_fullsized":"2024-09-11T05:20:17.014484+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12267,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14746","reported_seq":14883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.053418+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:17.053418+0000","last_peered":"2024-09-11T05:20:17.053418+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:17.053418+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22110208,"num_objects":5398,"num_object_clones":0,"num_object_copies":10796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11779,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14307","reported_seq":14446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.998436+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:16.998436+0000","last_peered":"2024-09-11T05:20:16.998436+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:16.998436+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11173,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14362","reported_seq":14502,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.027551+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:17.027551+0000","last_peered":"2024-09-11T05:20:17.027551+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:17.027551+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22274048,"num_objects":5438,"num_object_clones":0,"num_object_copies":10876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11120,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1894}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14329","reported_seq":14478,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.015586+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:17.015586+0000","last_peered":"2024-09-11T05:20:17.015586+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:17.015586+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549673,"num_objects":5262,"num_object_clones":0,"num_object_copies":10524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11336,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":354411137,"num_objects":86528,"num_object_clones":0,"num_object_copies":173056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195167,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8032,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885989376,"data_stored":885911652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586156,"kb_used_data":287900,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93785684,"statfs":{"total":96636764160,"available":96036540416,"internally_reserved":0,"allocated":294809600,"data_stored":294540229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584000,"kb_used_data":263536,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93787840,"statfs":{"total":96636764160,"available":96038748160,"internally_reserved":0,"allocated":269860864,"data_stored":269528273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215197,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712780,"kb_used_data":316024,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93659060,"statfs":{"total":96636764160,"available":95906877440,"internally_reserved":0,"allocated":323608576,"data_stored":323339200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322908160,"data_stored":322904681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269443072,"data_stored":269376378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293638144,"data_stored":293630593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:17.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:17.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14762", 'reported_seq': 14846, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:17.052002+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:17.052002+0000', 'last_peered': '2024-09-11T05:20:17.052002+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:17.052002+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22233088, 'num_objects': 5428, 'num_object_clones': 0, 'num_object_copies': 10856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14762, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1753}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:17.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:17.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14359", 'reported_seq': 16389, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:17.066681+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:17.066681+0000', 'last_peered': '2024-09-11T05:20:17.066681+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:17.066681+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21946392, 'num_objects': 5359, 'num_object_clones': 0, 'num_object_copies': 10718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5359, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12762, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 949, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2813}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:17.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14669", 'reported_seq': 14807, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:16.996978+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:16.996978+0000', 'last_peered': '2024-09-11T05:20:16.996978+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:16.996978+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22663168, 'num_objects': 5533, 'num_object_clones': 0, 'num_object_copies': 11066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11834, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14551", 'reported_seq': 14688, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:17.032432+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:17.032432+0000', 'last_peered': '2024-09-11T05:20:17.032432+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:17.032432+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22200320, 'num_objects': 5420, 'num_object_clones': 0, 'num_object_copies': 10840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12619, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1895}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14265", 'reported_seq': 14435, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:16.982194+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:16.982194+0000', 'last_peered': '2024-09-11T05:20:16.982194+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:16.982194+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22147072, 'num_objects': 5407, 'num_object_clones': 0, 'num_object_copies': 10814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11018, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1965}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14526", 'reported_seq': 14649, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:17.087341+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:17.087341+0000', 'last_peered': '2024-09-11T05:20:17.087341+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:17.087341+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22556672, 'num_objects': 5507, 'num_object_clones': 0, 'num_object_copies': 11014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12555, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1985}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14746", 'reported_seq': 14883, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:17.053418+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:17.053418+0000', 'last_peered': '2024-09-11T05:20:17.053418+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:17.053418+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22110208, 'num_objects': 5398, 'num_object_clones': 0, 'num_object_copies': 10796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5398, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11779, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1775}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:17.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14307", 'reported_seq': 14446, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:16.998436+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:16.998436+0000', 'last_peered': '2024-09-11T05:20:16.998436+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:16.998436+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21901312, 'num_objects': 5347, 'num_object_clones': 0, 'num_object_copies': 10694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11173, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1852}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:17.671 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:17.671 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14362", 'reported_seq': 14502, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:17.027551+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:17.027551+0000', 'last_peered': '2024-09-11T05:20:17.027551+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:17.027551+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22274048, 'num_objects': 5438, 'num_object_clones': 0, 'num_object_copies': 10876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5438, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11120, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1894}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:17.671 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:17.671 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14329", 'reported_seq': 14478, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:17.015586+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:17.015586+0000', 'last_peered': '2024-09-11T05:20:17.015586+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:17.015586+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21549673, 'num_objects': 5262, 'num_object_clones': 0, 'num_object_copies': 10524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11336, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:17.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:18.704 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:18.704 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34480/173060 objects degraded (19.924%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":496,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86530,"data_bytes":354411156,"bytes_used":1928126464,"bytes_avail":287982166016,"bytes_total":289910292480,"degraded_objects":34480,"degraded_total":173060,"degraded_ratio":0.19923725875418932,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":552},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:18.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:18.761 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:19.551 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:19.588 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-11T05:20:17.569832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":354411156,"num_objects":86530,"num_object_clones":0,"num_object_copies":173060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86530,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195169,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8033,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1882936,"kb_used_data":867460,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281232584,"statfs":{"total":289910292480,"available":287982166016,"internally_reserved":0,"allocated":888279040,"data_stored":887407702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1039780865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,12,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27176960,"num_objects":-6635,"num_object_clones":0,"num_object_copies":-13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006037"},"pg_stats":[{"pgid":"2.f","version":"47'14762","reported_seq":14846,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.052002+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:17.052002+0000","last_peered":"2024-09-11T05:20:17.052002+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:17.052002+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22233088,"num_objects":5428,"num_object_clones":0,"num_object_copies":10856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1753}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14530","reported_seq":14638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.077883+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:17.077883+0000","last_peered":"2024-09-11T05:20:17.077883+0000","last_clean":"2024-09-11T05:20:17.077883+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:17.077883+0000","last_undegraded":"2024-09-11T05:20:17.077883+0000","last_fullsized":"2024-09-11T05:20:17.077883+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14359","reported_seq":16389,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.066681+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:17.066681+0000","last_peered":"2024-09-11T05:20:17.066681+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:17.066681+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21946392,"num_objects":5359,"num_object_clones":0,"num_object_copies":10718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12762,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":949,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2813}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14632","reported_seq":14709,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:16.984448+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:16.984448+0000","last_peered":"2024-09-11T05:20:16.984448+0000","last_clean":"2024-09-11T05:20:16.984448+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:16.984448+0000","last_undegraded":"2024-09-11T05:20:16.984448+0000","last_fullsized":"2024-09-11T05:20:16.984448+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14632,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14704","reported_seq":14839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.004750+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:17.004750+0000","last_peered":"2024-09-11T05:20:17.004750+0000","last_clean":"2024-09-11T05:20:17.004750+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:17.004750+0000","last_undegraded":"2024-09-11T05:20:17.004750+0000","last_fullsized":"2024-09-11T05:20:17.004750+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22384640,"num_objects":5465,"num_object_clones":0,"num_object_copies":10930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11538,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14550","reported_seq":21901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:14.311007+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:14.311007+0000","last_peered":"2024-09-11T05:20:14.311007+0000","last_clean":"2024-09-11T05:20:14.311007+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:14.311007+0000","last_undegraded":"2024-09-11T05:20:14.311007+0000","last_fullsized":"2024-09-11T05:20:14.311007+0000","mapping_epoch":46,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22466560,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11094,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14669","reported_seq":14807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.996978+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:16.996978+0000","last_peered":"2024-09-11T05:20:16.996978+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:16.996978+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11834,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14447","reported_seq":14576,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.009569+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:17.009569+0000","last_peered":"2024-09-11T05:20:17.009569+0000","last_clean":"2024-09-11T05:20:17.009569+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:17.009569+0000","last_undegraded":"2024-09-11T05:20:17.009569+0000","last_fullsized":"2024-09-11T05:20:17.009569+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11750,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14551","reported_seq":14688,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.032432+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:17.032432+0000","last_peered":"2024-09-11T05:20:17.032432+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:17.032432+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12619,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1895}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14265","reported_seq":14435,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.982194+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:16.982194+0000","last_peered":"2024-09-11T05:20:16.982194+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:16.982194+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11018,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1965}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14526","reported_seq":14649,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.087341+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:17.087341+0000","last_peered":"2024-09-11T05:20:17.087341+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:17.087341+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12555,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1985}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14364","reported_seq":21455,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.014484+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:17.014484+0000","last_peered":"2024-09-11T05:20:17.014484+0000","last_clean":"2024-09-11T05:20:17.014484+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:17.014484+0000","last_undegraded":"2024-09-11T05:20:17.014484+0000","last_fullsized":"2024-09-11T05:20:17.014484+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12267,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14746","reported_seq":14883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.053418+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:17.053418+0000","last_peered":"2024-09-11T05:20:17.053418+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:17.053418+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22110208,"num_objects":5398,"num_object_clones":0,"num_object_copies":10796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11779,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14307","reported_seq":14446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.998436+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:16.998436+0000","last_peered":"2024-09-11T05:20:16.998436+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:16.998436+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11173,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14362","reported_seq":14502,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.027551+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:17.027551+0000","last_peered":"2024-09-11T05:20:17.027551+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:17.027551+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22274048,"num_objects":5438,"num_object_clones":0,"num_object_copies":10876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11120,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1894}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14329","reported_seq":14478,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.015586+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:17.015586+0000","last_peered":"2024-09-11T05:20:17.015586+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:17.015586+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549673,"num_objects":5262,"num_object_clones":0,"num_object_copies":10524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11336,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":354411137,"num_objects":86528,"num_object_clones":0,"num_object_copies":173056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195167,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8032,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885989376,"data_stored":885911652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586156,"kb_used_data":287900,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93785684,"statfs":{"total":96636764160,"available":96036540416,"internally_reserved":0,"allocated":294809600,"data_stored":294540229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584000,"kb_used_data":263536,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93787840,"statfs":{"total":96636764160,"available":96038748160,"internally_reserved":0,"allocated":269860864,"data_stored":269528273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215197,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712780,"kb_used_data":316024,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93659060,"statfs":{"total":96636764160,"available":95906877440,"internally_reserved":0,"allocated":323608576,"data_stored":323339200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322908160,"data_stored":322904681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":269443072,"data_stored":269376378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293638144,"data_stored":293630593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:22.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:23.515 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:23.565 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-11T05:20:21.570861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353931924,"num_objects":86413,"num_object_clones":0,"num_object_copies":172826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195286,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8033,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1884596,"kb_used_data":860800,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281230924,"statfs":{"total":289910292480,"available":287980466176,"internally_reserved":0,"allocated":881459200,"data_stored":880587862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1048300545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,5,9,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18571264,"num_objects":-4534,"num_object_clones":0,"num_object_copies":-9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4534,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005470"},"pg_stats":[{"pgid":"2.f","version":"47'14762","reported_seq":14846,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.052002+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:17.052002+0000","last_peered":"2024-09-11T05:20:17.052002+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:17.052002+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22233088,"num_objects":5428,"num_object_clones":0,"num_object_copies":10856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1753}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14530","reported_seq":14638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.077883+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:17.077883+0000","last_peered":"2024-09-11T05:20:17.077883+0000","last_clean":"2024-09-11T05:20:17.077883+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:17.077883+0000","last_undegraded":"2024-09-11T05:20:17.077883+0000","last_fullsized":"2024-09-11T05:20:17.077883+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14359","reported_seq":16389,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.066681+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:17.066681+0000","last_peered":"2024-09-11T05:20:17.066681+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:17.066681+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21946392,"num_objects":5359,"num_object_clones":0,"num_object_copies":10718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12762,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":949,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2813}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14632","reported_seq":14709,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:16.984448+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:16.984448+0000","last_peered":"2024-09-11T05:20:16.984448+0000","last_clean":"2024-09-11T05:20:16.984448+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:16.984448+0000","last_undegraded":"2024-09-11T05:20:16.984448+0000","last_fullsized":"2024-09-11T05:20:16.984448+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14632,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14704","reported_seq":14839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.004750+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:17.004750+0000","last_peered":"2024-09-11T05:20:17.004750+0000","last_clean":"2024-09-11T05:20:17.004750+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:17.004750+0000","last_undegraded":"2024-09-11T05:20:17.004750+0000","last_fullsized":"2024-09-11T05:20:17.004750+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22384640,"num_objects":5465,"num_object_clones":0,"num_object_copies":10930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11538,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14667","reported_seq":22017,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:19.451915+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:19.451915+0000","last_peered":"2024-09-11T05:20:19.451915+0000","last_clean":"2024-09-11T05:20:19.451915+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:19.451915+0000","last_undegraded":"2024-09-11T05:20:19.451915+0000","last_fullsized":"2024-09-11T05:20:19.451915+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21987328,"num_objects":5368,"num_object_clones":0,"num_object_copies":10736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11211,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14669","reported_seq":14807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.996978+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:16.996978+0000","last_peered":"2024-09-11T05:20:16.996978+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:16.996978+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11834,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14447","reported_seq":14576,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.009569+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:17.009569+0000","last_peered":"2024-09-11T05:20:17.009569+0000","last_clean":"2024-09-11T05:20:17.009569+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:17.009569+0000","last_undegraded":"2024-09-11T05:20:17.009569+0000","last_fullsized":"2024-09-11T05:20:17.009569+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11750,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14551","reported_seq":14688,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.032432+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:17.032432+0000","last_peered":"2024-09-11T05:20:17.032432+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:17.032432+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12619,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1895}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14265","reported_seq":14435,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.982194+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:16.982194+0000","last_peered":"2024-09-11T05:20:16.982194+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:16.982194+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11018,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1965}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14526","reported_seq":14649,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.087341+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:17.087341+0000","last_peered":"2024-09-11T05:20:17.087341+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:17.087341+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12555,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1985}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14364","reported_seq":21455,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:17.014484+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:17.014484+0000","last_peered":"2024-09-11T05:20:17.014484+0000","last_clean":"2024-09-11T05:20:17.014484+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:17.014484+0000","last_undegraded":"2024-09-11T05:20:17.014484+0000","last_fullsized":"2024-09-11T05:20:17.014484+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12267,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14746","reported_seq":14883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.053418+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:17.053418+0000","last_peered":"2024-09-11T05:20:17.053418+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:17.053418+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22110208,"num_objects":5398,"num_object_clones":0,"num_object_copies":10796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11779,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14307","reported_seq":14446,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:16.998436+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:16.998436+0000","last_peered":"2024-09-11T05:20:16.998436+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:16.998436+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11173,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14362","reported_seq":14502,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.027551+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:17.027551+0000","last_peered":"2024-09-11T05:20:17.027551+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:17.027551+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22274048,"num_objects":5438,"num_object_clones":0,"num_object_copies":10876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11120,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1894}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14329","reported_seq":14478,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:17.015586+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:17.015586+0000","last_peered":"2024-09-11T05:20:17.015586+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:17.015586+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549673,"num_objects":5262,"num_object_clones":0,"num_object_copies":10524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11336,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":353931905,"num_objects":86411,"num_object_clones":0,"num_object_copies":172822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195284,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8032,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":879005696,"data_stored":878927972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586156,"kb_used_data":287900,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93785684,"statfs":{"total":96636764160,"available":96036540416,"internally_reserved":0,"allocated":294809600,"data_stored":294540229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585660,"kb_used_data":256876,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93786180,"statfs":{"total":96636764160,"available":96037048320,"internally_reserved":0,"allocated":263041024,"data_stored":262708433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215197,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712780,"kb_used_data":316024,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93659060,"statfs":{"total":96636764160,"available":95906877440,"internally_reserved":0,"allocated":323608576,"data_stored":323339200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322908160,"data_stored":322904681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262459392,"data_stored":262392698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293638144,"data_stored":293630593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:23.566 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:24.351 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:24.398 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-11T05:20:23.571280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345043604,"num_objects":84243,"num_object_clones":0,"num_object_copies":168486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197456,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8123,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1881712,"kb_used_data":849592,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281233808,"statfs":{"total":289910292480,"available":287983419392,"internally_reserved":0,"allocated":869982208,"data_stored":869110870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1056820225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,8,6,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27459584,"num_objects":-6704,"num_object_clones":0,"num_object_copies":-13408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005413"},"pg_stats":[{"pgid":"2.f","version":"47'14921","reported_seq":15005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:21.948408+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:21.948408+0000","last_peered":"2024-09-11T05:20:21.948408+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:21.948408+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21581824,"num_objects":5269,"num_object_clones":0,"num_object_copies":10538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1594}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14666","reported_seq":14774,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.096330+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:22.096330+0000","last_peered":"2024-09-11T05:20:22.096330+0000","last_clean":"2024-09-11T05:20:22.096330+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:22.096330+0000","last_undegraded":"2024-09-11T05:20:22.096330+0000","last_fullsized":"2024-09-11T05:20:22.096330+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14509","reported_seq":16719,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.048664+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:22.048664+0000","last_peered":"2024-09-11T05:20:22.048664+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:22.048664+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21331992,"num_objects":5209,"num_object_clones":0,"num_object_copies":10418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12912,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14774","reported_seq":14851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.078583+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:22.078583+0000","last_peered":"2024-09-11T05:20:22.078583+0000","last_clean":"2024-09-11T05:20:22.078583+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:22.078583+0000","last_undegraded":"2024-09-11T05:20:22.078583+0000","last_fullsized":"2024-09-11T05:20:22.078583+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21241856,"num_objects":5186,"num_object_clones":0,"num_object_copies":10372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14774,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14843","reported_seq":14978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.005151+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:22.005151+0000","last_peered":"2024-09-11T05:20:22.005151+0000","last_clean":"2024-09-11T05:20:22.005151+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:22.005151+0000","last_undegraded":"2024-09-11T05:20:22.005151+0000","last_fullsized":"2024-09-11T05:20:22.005151+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21815296,"num_objects":5326,"num_object_clones":0,"num_object_copies":10652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11677,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14667","reported_seq":22017,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:19.451915+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:19.451915+0000","last_peered":"2024-09-11T05:20:19.451915+0000","last_clean":"2024-09-11T05:20:19.451915+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:19.451915+0000","last_undegraded":"2024-09-11T05:20:19.451915+0000","last_fullsized":"2024-09-11T05:20:19.451915+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21987328,"num_objects":5368,"num_object_clones":0,"num_object_copies":10736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11211,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14807","reported_seq":14943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.056161+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:22.056161+0000","last_peered":"2024-09-11T05:20:22.056161+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:22.056161+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22097920,"num_objects":5395,"num_object_clones":0,"num_object_copies":10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11972,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14587","reported_seq":14716,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:21.915942+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:21.915942+0000","last_peered":"2024-09-11T05:20:21.915942+0000","last_clean":"2024-09-11T05:20:21.915942+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:21.915942+0000","last_undegraded":"2024-09-11T05:20:21.915942+0000","last_fullsized":"2024-09-11T05:20:21.915942+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21696512,"num_objects":5297,"num_object_clones":0,"num_object_copies":10594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14693","reported_seq":14830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.049546+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:22.049546+0000","last_peered":"2024-09-11T05:20:22.049546+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:22.049546+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21618688,"num_objects":5278,"num_object_clones":0,"num_object_copies":10556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12761,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14425","reported_seq":14595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.069294+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:22.069294+0000","last_peered":"2024-09-11T05:20:22.069294+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:22.069294+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21491712,"num_objects":5247,"num_object_clones":0,"num_object_copies":10494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14679","reported_seq":14799,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.058279+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:22.058279+0000","last_peered":"2024-09-11T05:20:22.058279+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:22.058279+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21929984,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12708,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14498","reported_seq":21588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.125414+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:22.125414+0000","last_peered":"2024-09-11T05:20:22.125414+0000","last_clean":"2024-09-11T05:20:22.125414+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:22.125414+0000","last_undegraded":"2024-09-11T05:20:22.125414+0000","last_fullsized":"2024-09-11T05:20:22.125414+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12401,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14897","reported_seq":15033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.098303+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:22.098303+0000","last_peered":"2024-09-11T05:20:22.098303+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:22.098303+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21491712,"num_objects":5247,"num_object_clones":0,"num_object_copies":10494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11930,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14442","reported_seq":14581,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.106281+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:22.106281+0000","last_peered":"2024-09-11T05:20:22.106281+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:22.106281+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14522","reported_seq":14658,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:21.997823+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:21.997823+0000","last_peered":"2024-09-11T05:20:21.997823+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:21.997823+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21618688,"num_objects":5278,"num_object_clones":0,"num_object_copies":10556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11280,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1734}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14460","reported_seq":14609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.064296+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:22.064296+0000","last_peered":"2024-09-11T05:20:22.064296+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:22.064296+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21013097,"num_objects":5131,"num_object_clones":0,"num_object_copies":10262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345043585,"num_objects":84241,"num_object_clones":0,"num_object_copies":168482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84241,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197454,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8122,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867393536,"data_stored":867315812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585816,"kb_used_data":283400,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93786024,"statfs":{"total":96636764160,"available":96036888576,"internally_reserved":0,"allocated":290201600,"data_stored":289932229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585660,"kb_used_data":256876,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93786180,"statfs":{"total":96636764160,"available":96037048320,"internally_reserved":0,"allocated":263041024,"data_stored":262708433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215198,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710236,"kb_used_data":309316,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93661604,"statfs":{"total":96636764160,"available":95909482496,"internally_reserved":0,"allocated":316739584,"data_stored":316470208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315944960,"data_stored":315941481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262459392,"data_stored":262392698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288989184,"data_stored":288981633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:24.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:24.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14921", 'reported_seq': 15005, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:21.948408+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:21.948408+0000', 'last_peered': '2024-09-11T05:20:21.948408+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:21.948408+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21581824, 'num_objects': 5269, 'num_object_clones': 0, 'num_object_copies': 10538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14921, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1594}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:24.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:24.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14509", 'reported_seq': 16719, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.048664+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:22.048664+0000', 'last_peered': '2024-09-11T05:20:22.048664+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:22.048664+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21331992, 'num_objects': 5209, 'num_object_clones': 0, 'num_object_copies': 10418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5209, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12912, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1039, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2753}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:24.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:24.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14807", 'reported_seq': 14943, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.056161+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:22.056161+0000', 'last_peered': '2024-09-11T05:20:22.056161+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:22.056161+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22097920, 'num_objects': 5395, 'num_object_clones': 0, 'num_object_copies': 10790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11972, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1819}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:24.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14693", 'reported_seq': 14830, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.049546+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:22.049546+0000', 'last_peered': '2024-09-11T05:20:22.049546+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:22.049546+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21618688, 'num_objects': 5278, 'num_object_clones': 0, 'num_object_copies': 10556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5278, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12761, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1753}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14425", 'reported_seq': 14595, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.069294+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:22.069294+0000', 'last_peered': '2024-09-11T05:20:22.069294+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:22.069294+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21491712, 'num_objects': 5247, 'num_object_clones': 0, 'num_object_copies': 10494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11178, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1805}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14679", 'reported_seq': 14799, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.058279+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:22.058279+0000', 'last_peered': '2024-09-11T05:20:22.058279+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:22.058279+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21929984, 'num_objects': 5354, 'num_object_clones': 0, 'num_object_copies': 10708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12708, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1832}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14897", 'reported_seq': 15033, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.098303+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:22.098303+0000', 'last_peered': '2024-09-11T05:20:22.098303+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:22.098303+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21491712, 'num_objects': 5247, 'num_object_clones': 0, 'num_object_copies': 10494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11930, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1624}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14442", 'reported_seq': 14581, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.106281+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:22.106281+0000', 'last_peered': '2024-09-11T05:20:22.106281+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:22.106281+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21348352, 'num_objects': 5212, 'num_object_clones': 0, 'num_object_copies': 10424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5212, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11308, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1717}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:24.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:24.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14522", 'reported_seq': 14658, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:21.997823+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:21.997823+0000', 'last_peered': '2024-09-11T05:20:21.997823+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:21.997823+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21618688, 'num_objects': 5278, 'num_object_clones': 0, 'num_object_copies': 10556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5278, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11280, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1734}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:24.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:24.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14460", 'reported_seq': 14609, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:22.064296+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:22.064296+0000', 'last_peered': '2024-09-11T05:20:22.064296+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:22.064296+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21013097, 'num_objects': 5131, 'num_object_clones': 0, 'num_object_copies': 10262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11467, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1616}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:24.402 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:25.545 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:25.545 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34390/168486 objects degraded (20.411%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84243,"data_bytes":345043604,"bytes_used":1926873088,"bytes_avail":287983419392,"bytes_total":289910292480,"degraded_objects":34390,"degraded_total":168486,"degraded_ratio":0.2041119143430315,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":558},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:25.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:25.605 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:26.543 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:26.544 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:26.578 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-11T05:20:25.571804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":344437396,"num_objects":84095,"num_object_clones":0,"num_object_copies":168190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197604,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8123,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160841,"ondisk_log_size":160841,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1862448,"kb_used_data":842796,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281253072,"statfs":{"total":289910292480,"available":288003145728,"internally_reserved":0,"allocated":863023104,"data_stored":862151766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1044040705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,6,7,7,15,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19427328,"num_objects":-4743,"num_object_clones":0,"num_object_copies":-9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4743,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005504"},"pg_stats":[{"pgid":"2.f","version":"47'14921","reported_seq":15005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:21.948408+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:21.948408+0000","last_peered":"2024-09-11T05:20:21.948408+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:21.948408+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21581824,"num_objects":5269,"num_object_clones":0,"num_object_copies":10538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1594}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14666","reported_seq":14774,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.096330+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:22.096330+0000","last_peered":"2024-09-11T05:20:22.096330+0000","last_clean":"2024-09-11T05:20:22.096330+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:22.096330+0000","last_undegraded":"2024-09-11T05:20:22.096330+0000","last_fullsized":"2024-09-11T05:20:22.096330+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13064,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14509","reported_seq":16719,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.048664+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:22.048664+0000","last_peered":"2024-09-11T05:20:22.048664+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:22.048664+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21331992,"num_objects":5209,"num_object_clones":0,"num_object_copies":10418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12912,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14774","reported_seq":14851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.078583+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:22.078583+0000","last_peered":"2024-09-11T05:20:22.078583+0000","last_clean":"2024-09-11T05:20:22.078583+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:22.078583+0000","last_undegraded":"2024-09-11T05:20:22.078583+0000","last_fullsized":"2024-09-11T05:20:22.078583+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21241856,"num_objects":5186,"num_object_clones":0,"num_object_copies":10372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14774,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14843","reported_seq":14978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.005151+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:22.005151+0000","last_peered":"2024-09-11T05:20:22.005151+0000","last_clean":"2024-09-11T05:20:22.005151+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:22.005151+0000","last_undegraded":"2024-09-11T05:20:22.005151+0000","last_fullsized":"2024-09-11T05:20:22.005151+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21815296,"num_objects":5326,"num_object_clones":0,"num_object_copies":10652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11677,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14815","reported_seq":22166,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:24.523903+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:24.523903+0000","last_peered":"2024-09-11T05:20:24.523903+0000","last_clean":"2024-09-11T05:20:24.523903+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:24.523903+0000","last_undegraded":"2024-09-11T05:20:24.523903+0000","last_fullsized":"2024-09-11T05:20:24.523903+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21381120,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11359,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14807","reported_seq":14943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.056161+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:22.056161+0000","last_peered":"2024-09-11T05:20:22.056161+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:22.056161+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22097920,"num_objects":5395,"num_object_clones":0,"num_object_copies":10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11972,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14587","reported_seq":14716,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:21.915942+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:21.915942+0000","last_peered":"2024-09-11T05:20:21.915942+0000","last_clean":"2024-09-11T05:20:21.915942+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:21.915942+0000","last_undegraded":"2024-09-11T05:20:21.915942+0000","last_fullsized":"2024-09-11T05:20:21.915942+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21696512,"num_objects":5297,"num_object_clones":0,"num_object_copies":10594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14693","reported_seq":14830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.049546+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:22.049546+0000","last_peered":"2024-09-11T05:20:22.049546+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:22.049546+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21618688,"num_objects":5278,"num_object_clones":0,"num_object_copies":10556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12761,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14425","reported_seq":14595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.069294+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:22.069294+0000","last_peered":"2024-09-11T05:20:22.069294+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:22.069294+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21491712,"num_objects":5247,"num_object_clones":0,"num_object_copies":10494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14679","reported_seq":14799,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.058279+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:22.058279+0000","last_peered":"2024-09-11T05:20:22.058279+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:22.058279+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21929984,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12708,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14498","reported_seq":21588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:22.125414+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:22.125414+0000","last_peered":"2024-09-11T05:20:22.125414+0000","last_clean":"2024-09-11T05:20:22.125414+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:22.125414+0000","last_undegraded":"2024-09-11T05:20:22.125414+0000","last_fullsized":"2024-09-11T05:20:22.125414+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340160,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12401,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14897","reported_seq":15033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.098303+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:22.098303+0000","last_peered":"2024-09-11T05:20:22.098303+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:22.098303+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21491712,"num_objects":5247,"num_object_clones":0,"num_object_copies":10494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11930,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14442","reported_seq":14581,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.106281+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:22.106281+0000","last_peered":"2024-09-11T05:20:22.106281+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:22.106281+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14522","reported_seq":14658,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:21.997823+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:21.997823+0000","last_peered":"2024-09-11T05:20:21.997823+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:21.997823+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21618688,"num_objects":5278,"num_object_clones":0,"num_object_copies":10556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11280,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1734}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14460","reported_seq":14609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:22.064296+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:22.064296+0000","last_peered":"2024-09-11T05:20:22.064296+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:22.064296+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21013097,"num_objects":5131,"num_object_clones":0,"num_object_copies":10262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":344437377,"num_objects":84093,"num_object_clones":0,"num_object_copies":168186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197602,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8122,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860250112,"data_stored":860172388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160838,"ondisk_log_size":160838,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585816,"kb_used_data":283400,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93786024,"statfs":{"total":96636764160,"available":96036888576,"internally_reserved":0,"allocated":290201600,"data_stored":289932229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566396,"kb_used_data":250080,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93805444,"statfs":{"total":96636764160,"available":96056774656,"internally_reserved":0,"allocated":256081920,"data_stored":255749329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,4,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215198,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710236,"kb_used_data":309316,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93661604,"statfs":{"total":96636764160,"available":95909482496,"internally_reserved":0,"allocated":316739584,"data_stored":316470208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315944960,"data_stored":315941481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255315968,"data_stored":255249274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288989184,"data_stored":288981633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:29.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:30.416 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:30.417 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:30.483 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T05:20:29.572751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":335856276,"num_objects":82000,"num_object_clones":0,"num_object_copies":164000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82000,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199699,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8223,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1847056,"kb_used_data":831568,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281268464,"statfs":{"total":289910292480,"available":288018907136,"internally_reserved":0,"allocated":851525632,"data_stored":850654294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1039780865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,8,8,9,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18554880,"num_objects":-4530,"num_object_clones":0,"num_object_copies":-9060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4530,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002918"},"pg_stats":[{"pgid":"2.f","version":"47'15051","reported_seq":15135,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.079823+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:27.079823+0000","last_peered":"2024-09-11T05:20:27.079823+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:27.079823+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15051,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14828","reported_seq":14936,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.096258+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:27.096258+0000","last_peered":"2024-09-11T05:20:27.096258+0000","last_clean":"2024-09-11T05:20:27.096258+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:27.096258+0000","last_undegraded":"2024-09-11T05:20:27.096258+0000","last_fullsized":"2024-09-11T05:20:27.096258+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13226,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14646","reported_seq":17055,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.116615+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:27.116615+0000","last_peered":"2024-09-11T05:20:27.116615+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:27.116615+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770840,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13049,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14902","reported_seq":14979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.030991+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:27.030991+0000","last_peered":"2024-09-11T05:20:27.030991+0000","last_clean":"2024-09-11T05:20:27.030991+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:27.030991+0000","last_undegraded":"2024-09-11T05:20:27.030991+0000","last_fullsized":"2024-09-11T05:20:27.030991+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14902,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14980","reported_seq":15115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.121011+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:27.121011+0000","last_peered":"2024-09-11T05:20:27.121011+0000","last_clean":"2024-09-11T05:20:27.121011+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:27.121011+0000","last_undegraded":"2024-09-11T05:20:27.121011+0000","last_fullsized":"2024-09-11T05:20:27.121011+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21254144,"num_objects":5189,"num_object_clones":0,"num_object_copies":10378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14815","reported_seq":22166,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:24.523903+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:24.523903+0000","last_peered":"2024-09-11T05:20:24.523903+0000","last_clean":"2024-09-11T05:20:24.523903+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:24.523903+0000","last_undegraded":"2024-09-11T05:20:24.523903+0000","last_fullsized":"2024-09-11T05:20:24.523903+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21381120,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11359,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14943","reported_seq":15080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.144494+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:27.144494+0000","last_peered":"2024-09-11T05:20:27.144494+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:27.144494+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21540864,"num_objects":5259,"num_object_clones":0,"num_object_copies":10518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14739","reported_seq":14868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.106627+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:27.106627+0000","last_peered":"2024-09-11T05:20:27.106627+0000","last_clean":"2024-09-11T05:20:27.106627+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:27.106627+0000","last_undegraded":"2024-09-11T05:20:27.106627+0000","last_fullsized":"2024-09-11T05:20:27.106627+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21073920,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12042,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14832","reported_seq":14969,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.116474+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:27.116474+0000","last_peered":"2024-09-11T05:20:27.116474+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:27.116474+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12900,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14559","reported_seq":14730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.023656+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:27.023656+0000","last_peered":"2024-09-11T05:20:27.023656+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:27.023656+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20942848,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14849","reported_seq":14967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.099524+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:27.099524+0000","last_peered":"2024-09-11T05:20:27.099524+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:27.099524+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12878,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14635","reported_seq":21726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.003833+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:27.003833+0000","last_peered":"2024-09-11T05:20:27.003833+0000","last_clean":"2024-09-11T05:20:27.003833+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:27.003833+0000","last_undegraded":"2024-09-11T05:20:27.003833+0000","last_fullsized":"2024-09-11T05:20:27.003833+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20779008,"num_objects":5073,"num_object_clones":0,"num_object_copies":10146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15038","reported_seq":15175,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:26.987390+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:26.987390+0000","last_peered":"2024-09-11T05:20:26.987390+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:26.987390+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12071,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14573","reported_seq":14712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.133980+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:27.133980+0000","last_peered":"2024-09-11T05:20:27.133980+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:27.133980+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11439,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14650","reported_seq":14790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.106955+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:27.106955+0000","last_peered":"2024-09-11T05:20:27.106955+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:27.106955+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21094400,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11408,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14593","reported_seq":14742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.056730+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:27.056730+0000","last_peered":"2024-09-11T05:20:27.056730+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:27.056730+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20468329,"num_objects":4998,"num_object_clones":0,"num_object_copies":9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11600,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":335856257,"num_objects":81998,"num_object_clones":0,"num_object_copies":163996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81998,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199697,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8222,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":848691200,"data_stored":848613476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585444,"kb_used_data":278868,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93786396,"statfs":{"total":96636764160,"available":96037269504,"internally_reserved":0,"allocated":285560832,"data_stored":285291461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566396,"kb_used_data":250080,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93805444,"statfs":{"total":96636764160,"available":96056774656,"internally_reserved":0,"allocated":256081920,"data_stored":255749329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,4,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215199,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695216,"kb_used_data":302620,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93676624,"statfs":{"total":96636764160,"available":95924862976,"internally_reserved":0,"allocated":309882880,"data_stored":309613504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":309116928,"data_stored":309113449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255315968,"data_stored":255249274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284258304,"data_stored":284250753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:30.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:31.245 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:31.250 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:31.308 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T05:20:29.572751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":335856276,"num_objects":82000,"num_object_clones":0,"num_object_copies":164000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82000,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199699,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8223,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1847056,"kb_used_data":831568,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281268464,"statfs":{"total":289910292480,"available":288018907136,"internally_reserved":0,"allocated":851525632,"data_stored":850654294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1039780865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,8,8,9,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18554880,"num_objects":-4530,"num_object_clones":0,"num_object_copies":-9060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4530,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002918"},"pg_stats":[{"pgid":"2.f","version":"47'15051","reported_seq":15135,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.079823+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:27.079823+0000","last_peered":"2024-09-11T05:20:27.079823+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:27.079823+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15051,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14828","reported_seq":14936,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.096258+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:27.096258+0000","last_peered":"2024-09-11T05:20:27.096258+0000","last_clean":"2024-09-11T05:20:27.096258+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:27.096258+0000","last_undegraded":"2024-09-11T05:20:27.096258+0000","last_fullsized":"2024-09-11T05:20:27.096258+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13226,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14646","reported_seq":17055,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.116615+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:27.116615+0000","last_peered":"2024-09-11T05:20:27.116615+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:27.116615+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770840,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13049,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14902","reported_seq":14979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.030991+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:27.030991+0000","last_peered":"2024-09-11T05:20:27.030991+0000","last_clean":"2024-09-11T05:20:27.030991+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:27.030991+0000","last_undegraded":"2024-09-11T05:20:27.030991+0000","last_fullsized":"2024-09-11T05:20:27.030991+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14902,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14980","reported_seq":15115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.121011+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:27.121011+0000","last_peered":"2024-09-11T05:20:27.121011+0000","last_clean":"2024-09-11T05:20:27.121011+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:27.121011+0000","last_undegraded":"2024-09-11T05:20:27.121011+0000","last_fullsized":"2024-09-11T05:20:27.121011+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21254144,"num_objects":5189,"num_object_clones":0,"num_object_copies":10378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14815","reported_seq":22166,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:24.523903+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:24.523903+0000","last_peered":"2024-09-11T05:20:24.523903+0000","last_clean":"2024-09-11T05:20:24.523903+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:24.523903+0000","last_undegraded":"2024-09-11T05:20:24.523903+0000","last_fullsized":"2024-09-11T05:20:24.523903+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21381120,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11359,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14943","reported_seq":15080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.144494+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:27.144494+0000","last_peered":"2024-09-11T05:20:27.144494+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:27.144494+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21540864,"num_objects":5259,"num_object_clones":0,"num_object_copies":10518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14739","reported_seq":14868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.106627+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:27.106627+0000","last_peered":"2024-09-11T05:20:27.106627+0000","last_clean":"2024-09-11T05:20:27.106627+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:27.106627+0000","last_undegraded":"2024-09-11T05:20:27.106627+0000","last_fullsized":"2024-09-11T05:20:27.106627+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21073920,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12042,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14832","reported_seq":14969,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.116474+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:27.116474+0000","last_peered":"2024-09-11T05:20:27.116474+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:27.116474+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12900,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14559","reported_seq":14730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.023656+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:27.023656+0000","last_peered":"2024-09-11T05:20:27.023656+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:27.023656+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20942848,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14849","reported_seq":14967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.099524+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:27.099524+0000","last_peered":"2024-09-11T05:20:27.099524+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:27.099524+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12878,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707859+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707859+0000","last_peered":"2024-09-11T05:19:25.707859+0000","last_clean":"2024-09-11T05:19:25.707859+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707859+0000","last_undegraded":"2024-09-11T05:19:25.707859+0000","last_fullsized":"2024-09-11T05:19:25.707859+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14635","reported_seq":21726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.003833+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:27.003833+0000","last_peered":"2024-09-11T05:20:27.003833+0000","last_clean":"2024-09-11T05:20:27.003833+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:27.003833+0000","last_undegraded":"2024-09-11T05:20:27.003833+0000","last_fullsized":"2024-09-11T05:20:27.003833+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20779008,"num_objects":5073,"num_object_clones":0,"num_object_copies":10146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15038","reported_seq":15175,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:26.987390+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:26.987390+0000","last_peered":"2024-09-11T05:20:26.987390+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:26.987390+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12071,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708386+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708386+0000","last_peered":"2024-09-11T05:19:25.708386+0000","last_clean":"2024-09-11T05:19:25.708386+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708386+0000","last_undegraded":"2024-09-11T05:19:25.708386+0000","last_fullsized":"2024-09-11T05:19:25.708386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14573","reported_seq":14712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.133980+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:27.133980+0000","last_peered":"2024-09-11T05:20:27.133980+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:27.133980+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11439,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14650","reported_seq":14790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.106955+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:27.106955+0000","last_peered":"2024-09-11T05:20:27.106955+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:27.106955+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21094400,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11408,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708312+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708312+0000","last_peered":"2024-09-11T05:19:25.708312+0000","last_clean":"2024-09-11T05:19:25.708312+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708312+0000","last_undegraded":"2024-09-11T05:19:25.708312+0000","last_fullsized":"2024-09-11T05:19:25.708312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14593","reported_seq":14742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.056730+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:27.056730+0000","last_peered":"2024-09-11T05:20:27.056730+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:27.056730+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20468329,"num_objects":4998,"num_object_clones":0,"num_object_copies":9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11600,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":335856257,"num_objects":81998,"num_object_clones":0,"num_object_copies":163996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81998,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199697,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8222,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":848691200,"data_stored":848613476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585444,"kb_used_data":278868,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93786396,"statfs":{"total":96636764160,"available":96037269504,"internally_reserved":0,"allocated":285560832,"data_stored":285291461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566396,"kb_used_data":250080,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93805444,"statfs":{"total":96636764160,"available":96056774656,"internally_reserved":0,"allocated":256081920,"data_stored":255749329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,4,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215199,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695216,"kb_used_data":302620,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93676624,"statfs":{"total":96636764160,"available":95924862976,"internally_reserved":0,"allocated":309882880,"data_stored":309613504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":309116928,"data_stored":309113449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255315968,"data_stored":255249274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284258304,"data_stored":284250753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:31.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:31.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15051", 'reported_seq': 15135, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.079823+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:27.079823+0000', 'last_peered': '2024-09-11T05:20:27.079823+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:27.079823+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21049344, 'num_objects': 5139, 'num_object_clones': 0, 'num_object_copies': 10278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15051, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1464}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:31.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:31.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14646", 'reported_seq': 17055, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.116615+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:27.116615+0000', 'last_peered': '2024-09-11T05:20:27.116615+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:27.116615+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20770840, 'num_objects': 5072, 'num_object_clones': 0, 'num_object_copies': 10144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2356, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5072, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13049, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1139, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2716}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14943", 'reported_seq': 15080, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.144494+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:27.144494+0000', 'last_peered': '2024-09-11T05:20:27.144494+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:27.144494+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21540864, 'num_objects': 5259, 'num_object_clones': 0, 'num_object_copies': 10518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12108, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14832", 'reported_seq': 14969, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.116474+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:27.116474+0000', 'last_peered': '2024-09-11T05:20:27.116474+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:27.116474+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21049344, 'num_objects': 5139, 'num_object_clones': 0, 'num_object_copies': 10278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12900, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14559", 'reported_seq': 14730, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.023656+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:27.023656+0000', 'last_peered': '2024-09-11T05:20:27.023656+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:27.023656+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20942848, 'num_objects': 5113, 'num_object_clones': 0, 'num_object_copies': 10226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11312, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1671}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14849", 'reported_seq': 14967, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.099524+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:27.099524+0000', 'last_peered': '2024-09-11T05:20:27.099524+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:27.099524+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21233664, 'num_objects': 5184, 'num_object_clones': 0, 'num_object_copies': 10368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5184, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12878, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1662}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:31.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15038", 'reported_seq': 15175, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:26.987390+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:26.987390+0000', 'last_peered': '2024-09-11T05:20:26.987390+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:26.987390+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20914176, 'num_objects': 5106, 'num_object_clones': 0, 'num_object_copies': 10212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12071, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1483}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:31.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:31.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14573", 'reported_seq': 14712, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.133980+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:27.133980+0000', 'last_peered': '2024-09-11T05:20:27.133980+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:27.133980+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20811776, 'num_objects': 5081, 'num_object_clones': 0, 'num_object_copies': 10162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11439, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:31.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:31.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14650", 'reported_seq': 14790, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.106955+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:27.106955+0000', 'last_peered': '2024-09-11T05:20:27.106955+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:27.106955+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21094400, 'num_objects': 5150, 'num_object_clones': 0, 'num_object_copies': 10300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11408, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1606}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:31.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:31.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14593", 'reported_seq': 14742, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:27.056730+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:27.056730+0000', 'last_peered': '2024-09-11T05:20:27.056730+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:27.056730+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20468329, 'num_objects': 4998, 'num_object_clones': 0, 'num_object_copies': 9996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4998, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11600, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1483}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:31.311 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:32.241 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:32.241 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34290/163714 objects degraded (20.945%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":510,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":81857,"data_bytes":335270548,"bytes_used":1911558144,"bytes_avail":287998734336,"bytes_total":289910292480,"degraded_objects":34290,"degraded_total":163714,"degraded_ratio":0.20945062731348571,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:32.286 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:32.286 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:33.077 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:33.078 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:33.135 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-11T05:20:31.573168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":335270548,"num_objects":81857,"num_object_clones":0,"num_object_copies":163714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199842,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8223,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1866756,"kb_used_data":824912,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281248764,"statfs":{"total":289910292480,"available":287998734336,"internally_reserved":0,"allocated":844709888,"data_stored":843838550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1066781697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,8,11,10,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18661376,"num_objects":-4556,"num_object_clones":0,"num_object_copies":-9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002782"},"pg_stats":[{"pgid":"2.f","version":"47'15051","reported_seq":15135,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.079823+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:27.079823+0000","last_peered":"2024-09-11T05:20:27.079823+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:27.079823+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15051,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14828","reported_seq":14936,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.096258+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:27.096258+0000","last_peered":"2024-09-11T05:20:27.096258+0000","last_clean":"2024-09-11T05:20:27.096258+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:27.096258+0000","last_undegraded":"2024-09-11T05:20:27.096258+0000","last_fullsized":"2024-09-11T05:20:27.096258+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13226,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14646","reported_seq":17055,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.116615+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:27.116615+0000","last_peered":"2024-09-11T05:20:27.116615+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:27.116615+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770840,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13049,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14902","reported_seq":14979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.030991+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:27.030991+0000","last_peered":"2024-09-11T05:20:27.030991+0000","last_clean":"2024-09-11T05:20:27.030991+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:27.030991+0000","last_undegraded":"2024-09-11T05:20:27.030991+0000","last_fullsized":"2024-09-11T05:20:27.030991+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14902,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14980","reported_seq":15115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.121011+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:27.121011+0000","last_peered":"2024-09-11T05:20:27.121011+0000","last_clean":"2024-09-11T05:20:27.121011+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:27.121011+0000","last_undegraded":"2024-09-11T05:20:27.121011+0000","last_fullsized":"2024-09-11T05:20:27.121011+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21254144,"num_objects":5189,"num_object_clones":0,"num_object_copies":10378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14958","reported_seq":22308,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:29.541245+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:29.541245+0000","last_peered":"2024-09-11T05:20:29.541245+0000","last_clean":"2024-09-11T05:20:29.541245+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:29.541245+0000","last_undegraded":"2024-09-11T05:20:29.541245+0000","last_fullsized":"2024-09-11T05:20:29.541245+0000","mapping_epoch":46,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20795392,"num_objects":5077,"num_object_clones":0,"num_object_copies":10154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11502,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'14943","reported_seq":15080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.144494+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:27.144494+0000","last_peered":"2024-09-11T05:20:27.144494+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:27.144494+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21540864,"num_objects":5259,"num_object_clones":0,"num_object_copies":10518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14739","reported_seq":14868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.106627+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:27.106627+0000","last_peered":"2024-09-11T05:20:27.106627+0000","last_clean":"2024-09-11T05:20:27.106627+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:27.106627+0000","last_undegraded":"2024-09-11T05:20:27.106627+0000","last_fullsized":"2024-09-11T05:20:27.106627+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21073920,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12042,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14832","reported_seq":14969,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.116474+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:27.116474+0000","last_peered":"2024-09-11T05:20:27.116474+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:27.116474+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21049344,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12900,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14559","reported_seq":14730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.023656+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:27.023656+0000","last_peered":"2024-09-11T05:20:27.023656+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:27.023656+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20942848,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14849","reported_seq":14967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.099524+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:27.099524+0000","last_peered":"2024-09-11T05:20:27.099524+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:27.099524+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12878,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14635","reported_seq":21726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:27.003833+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:27.003833+0000","last_peered":"2024-09-11T05:20:27.003833+0000","last_clean":"2024-09-11T05:20:27.003833+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:27.003833+0000","last_undegraded":"2024-09-11T05:20:27.003833+0000","last_fullsized":"2024-09-11T05:20:27.003833+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20779008,"num_objects":5073,"num_object_clones":0,"num_object_copies":10146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15038","reported_seq":15175,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:26.987390+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:26.987390+0000","last_peered":"2024-09-11T05:20:26.987390+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:26.987390+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12071,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14573","reported_seq":14712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.133980+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:27.133980+0000","last_peered":"2024-09-11T05:20:27.133980+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:27.133980+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11439,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14650","reported_seq":14790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.106955+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:27.106955+0000","last_peered":"2024-09-11T05:20:27.106955+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:27.106955+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21094400,"num_objects":5150,"num_object_clones":0,"num_object_copies":10300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11408,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14593","reported_seq":14742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:27.056730+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:27.056730+0000","last_peered":"2024-09-11T05:20:27.056730+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:27.056730+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20468329,"num_objects":4998,"num_object_clones":0,"num_object_copies":9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11600,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":335270529,"num_objects":81855,"num_object_clones":0,"num_object_copies":163710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199840,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8222,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841637888,"data_stored":841560164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585444,"kb_used_data":278868,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93786396,"statfs":{"total":96636764160,"available":96037269504,"internally_reserved":0,"allocated":285560832,"data_stored":285291461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586096,"kb_used_data":243424,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93785744,"statfs":{"total":96636764160,"available":96036601856,"internally_reserved":0,"allocated":249266176,"data_stored":248933585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215199,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695216,"kb_used_data":302620,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93676624,"statfs":{"total":96636764160,"available":95924862976,"internally_reserved":0,"allocated":309882880,"data_stored":309613504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":309116928,"data_stored":309113449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":248262656,"data_stored":248195962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284258304,"data_stored":284250753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:36.138 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:36.834 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:36.834 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:36.895 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-11T05:20:35.574088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":325915284,"num_objects":79573,"num_object_clones":0,"num_object_copies":159146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202126,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8318,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160963,"ondisk_log_size":160963,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1860216,"kb_used_data":805896,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281255304,"statfs":{"total":289910292480,"available":288005431296,"internally_reserved":0,"allocated":825237504,"data_stored":824366166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1079561217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,11,9,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19128320,"num_objects":-4670,"num_object_clones":0,"num_object_copies":-9340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4670,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002807"},"pg_stats":[{"pgid":"2.f","version":"47'15205","reported_seq":15289,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.192839+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:32.192839+0000","last_peered":"2024-09-11T05:20:32.192839+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:32.192839+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20418560,"num_objects":4985,"num_object_clones":0,"num_object_copies":9970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15205,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14973","reported_seq":15081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:32.116616+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:32.116616+0000","last_peered":"2024-09-11T05:20:32.116616+0000","last_clean":"2024-09-11T05:20:32.116616+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:32.116616+0000","last_undegraded":"2024-09-11T05:20:32.116616+0000","last_fullsized":"2024-09-11T05:20:32.116616+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20180992,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13371,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14784","reported_seq":17383,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.120283+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:32.120283+0000","last_peered":"2024-09-11T05:20:32.120283+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:32.120283+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205592,"num_objects":4934,"num_object_clones":0,"num_object_copies":9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13187,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2673}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15048","reported_seq":15125,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:32.029759+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:32.029759+0000","last_peered":"2024-09-11T05:20:32.029759+0000","last_clean":"2024-09-11T05:20:32.029759+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:32.029759+0000","last_undegraded":"2024-09-11T05:20:32.029759+0000","last_fullsized":"2024-09-11T05:20:32.029759+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":4912,"num_object_clones":0,"num_object_copies":9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15048,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15121","reported_seq":15256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:31.990763+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:31.990763+0000","last_peered":"2024-09-11T05:20:31.990763+0000","last_clean":"2024-09-11T05:20:31.990763+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:31.990763+0000","last_undegraded":"2024-09-11T05:20:31.990763+0000","last_fullsized":"2024-09-11T05:20:31.990763+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20676608,"num_objects":5048,"num_object_clones":0,"num_object_copies":10096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11955,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15115","reported_seq":22466,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:34.674890+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:34.674890+0000","last_peered":"2024-09-11T05:20:34.674890+0000","last_clean":"2024-09-11T05:20:34.674890+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:34.674890+0000","last_undegraded":"2024-09-11T05:20:34.674890+0000","last_fullsized":"2024-09-11T05:20:34.674890+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20152320,"num_objects":4920,"num_object_clones":0,"num_object_copies":9840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15085","reported_seq":15223,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.019397+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:32.019397+0000","last_peered":"2024-09-11T05:20:32.019397+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:32.019397+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20959232,"num_objects":5117,"num_object_clones":0,"num_object_copies":10234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12250,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14884","reported_seq":15014,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:32.202406+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:32.202406+0000","last_peered":"2024-09-11T05:20:32.202406+0000","last_clean":"2024-09-11T05:20:32.202406+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:32.202406+0000","last_undegraded":"2024-09-11T05:20:32.202406+0000","last_fullsized":"2024-09-11T05:20:32.202406+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20480000,"num_objects":5000,"num_object_clones":0,"num_object_copies":10000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14962","reported_seq":15099,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:31.946097+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:31.946097+0000","last_peered":"2024-09-11T05:20:31.946097+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:31.946097+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13030,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1484}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14690","reported_seq":14861,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.129335+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:32.129335+0000","last_peered":"2024-09-11T05:20:32.129335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:32.129335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20406272,"num_objects":4982,"num_object_clones":0,"num_object_copies":9964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11443,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1540}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14997","reported_seq":15118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.104870+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:32.104870+0000","last_peered":"2024-09-11T05:20:32.104870+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:32.104870+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20627456,"num_objects":5036,"num_object_clones":0,"num_object_copies":10072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13026,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1514}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14773","reported_seq":21864,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:32.052550+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:32.052550+0000","last_peered":"2024-09-11T05:20:32.052550+0000","last_clean":"2024-09-11T05:20:32.052550+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:32.052550+0000","last_undegraded":"2024-09-11T05:20:32.052550+0000","last_fullsized":"2024-09-11T05:20:32.052550+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12676,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15187","reported_seq":15324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.129417+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:32.129417+0000","last_peered":"2024-09-11T05:20:32.129417+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:32.129417+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20303872,"num_objects":4957,"num_object_clones":0,"num_object_copies":9914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12220,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14715","reported_seq":14854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.199208+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:32.199208+0000","last_peered":"2024-09-11T05:20:32.199208+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:32.199208+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20230144,"num_objects":4939,"num_object_clones":0,"num_object_copies":9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11581,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14800","reported_seq":14941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.164266+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:32.164266+0000","last_peered":"2024-09-11T05:20:32.164266+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:32.164266+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20480000,"num_objects":5000,"num_object_clones":0,"num_object_copies":10000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14721","reported_seq":14870,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:32.092778+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:32.092778+0000","last_peered":"2024-09-11T05:20:32.092778+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:32.092778+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19944041,"num_objects":4870,"num_object_clones":0,"num_object_copies":9740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11728,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1355}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":325915265,"num_objects":79571,"num_object_clones":0,"num_object_copies":159142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202124,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8317,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823083008,"data_stored":823005284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160960,"ondisk_log_size":160960,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584260,"kb_used_data":273524,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93787580,"statfs":{"total":96636764160,"available":96038481920,"internally_reserved":0,"allocated":280088576,"data_stored":279819205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583592,"kb_used_data":236760,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93788248,"statfs":{"total":96636764160,"available":96039165952,"internally_reserved":0,"allocated":242442240,"data_stored":242109649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215200,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692364,"kb_used_data":295612,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93679476,"statfs":{"total":96636764160,"available":95927783424,"internally_reserved":0,"allocated":302706688,"data_stored":302437312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,6,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302059520,"data_stored":302056041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":241401856,"data_stored":241335162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279621632,"data_stored":279614081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:36.896 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:37.752 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:37.754 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:37.812 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-11T05:20:37.574531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317141652,"num_objects":77431,"num_object_clones":0,"num_object_copies":154862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204268,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8418,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1840496,"kb_used_data":794492,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281275024,"statfs":{"total":289910292480,"available":288025624576,"internally_reserved":0,"allocated":813559808,"data_stored":812688470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1071041537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,8,12,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27295744,"num_objects":-6664,"num_object_clones":0,"num_object_copies":-13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002726"},"pg_stats":[{"pgid":"2.f","version":"47'15336","reported_seq":15420,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.186415+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:37.186415+0000","last_peered":"2024-09-11T05:20:37.186415+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:37.186415+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15133","reported_seq":15241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.153087+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:37.153087+0000","last_peered":"2024-09-11T05:20:37.153087+0000","last_clean":"2024-09-11T05:20:37.153087+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:37.153087+0000","last_undegraded":"2024-09-11T05:20:37.153087+0000","last_fullsized":"2024-09-11T05:20:37.153087+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13531,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14914","reported_seq":17715,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.136894+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:37.136894+0000","last_peered":"2024-09-11T05:20:37.136894+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:37.136894+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673112,"num_objects":4804,"num_object_clones":0,"num_object_copies":9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13317,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2643}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15205","reported_seq":15282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.122932+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:37.122932+0000","last_peered":"2024-09-11T05:20:37.122932+0000","last_clean":"2024-09-11T05:20:37.122932+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:37.122932+0000","last_undegraded":"2024-09-11T05:20:37.122932+0000","last_fullsized":"2024-09-11T05:20:37.122932+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19476480,"num_objects":4755,"num_object_clones":0,"num_object_copies":9510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15205,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15281","reported_seq":15416,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.184636+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:37.184636+0000","last_peered":"2024-09-11T05:20:37.184636+0000","last_clean":"2024-09-11T05:20:37.184636+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:37.184636+0000","last_undegraded":"2024-09-11T05:20:37.184636+0000","last_fullsized":"2024-09-11T05:20:37.184636+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12115,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15115","reported_seq":22466,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:34.674890+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:34.674890+0000","last_peered":"2024-09-11T05:20:34.674890+0000","last_clean":"2024-09-11T05:20:34.674890+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:34.674890+0000","last_undegraded":"2024-09-11T05:20:34.674890+0000","last_fullsized":"2024-09-11T05:20:34.674890+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20152320,"num_objects":4920,"num_object_clones":0,"num_object_copies":9840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15223","reported_seq":15361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.193227+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:37.193227+0000","last_peered":"2024-09-11T05:20:37.193227+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:37.193227+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15023","reported_seq":15152,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.181338+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:37.181338+0000","last_peered":"2024-09-11T05:20:37.181338+0000","last_clean":"2024-09-11T05:20:37.181338+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:37.181338+0000","last_undegraded":"2024-09-11T05:20:37.181338+0000","last_fullsized":"2024-09-11T05:20:37.181338+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19910656,"num_objects":4861,"num_object_clones":0,"num_object_copies":9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12326,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15091","reported_seq":15228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.119383+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:37.119383+0000","last_peered":"2024-09-11T05:20:37.119383+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:37.119383+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988480,"num_objects":4880,"num_object_clones":0,"num_object_copies":9760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13159,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1355}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14837","reported_seq":15008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.140620+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:37.140620+0000","last_peered":"2024-09-11T05:20:37.140620+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:37.140620+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19804160,"num_objects":4835,"num_object_clones":0,"num_object_copies":9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15148","reported_seq":15268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.041306+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:37.041306+0000","last_peered":"2024-09-11T05:20:37.041306+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:37.041306+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13177,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14916","reported_seq":22007,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.122521+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:37.122521+0000","last_peered":"2024-09-11T05:20:37.122521+0000","last_clean":"2024-09-11T05:20:37.122521+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:37.122521+0000","last_undegraded":"2024-09-11T05:20:37.122521+0000","last_fullsized":"2024-09-11T05:20:37.122521+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628032,"num_objects":4792,"num_object_clones":0,"num_object_copies":9584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12819,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15315","reported_seq":15452,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.054057+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:37.054057+0000","last_peered":"2024-09-11T05:20:37.054057+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:37.054057+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19779584,"num_objects":4829,"num_object_clones":0,"num_object_copies":9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14851","reported_seq":14990,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.096749+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:37.096749+0000","last_peered":"2024-09-11T05:20:37.096749+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:37.096749+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673088,"num_objects":4803,"num_object_clones":0,"num_object_copies":9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14952","reported_seq":15093,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.112775+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:37.112775+0000","last_peered":"2024-09-11T05:20:37.112775+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:37.112775+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11710,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14862","reported_seq":15011,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.038326+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:37.038326+0000","last_peered":"2024-09-11T05:20:37.038326+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:37.038326+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19366505,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11869,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1214}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317141633,"num_objects":77429,"num_object_clones":0,"num_object_copies":154858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204266,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8417,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811212800,"data_stored":811135076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567220,"kb_used_data":268960,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93804620,"statfs":{"total":96636764160,"available":96055930880,"internally_reserved":0,"allocated":275415040,"data_stored":275145669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583592,"kb_used_data":236760,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93788248,"statfs":{"total":96636764160,"available":96039165952,"internally_reserved":0,"allocated":242442240,"data_stored":242109649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215201,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689684,"kb_used_data":288772,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93682156,"statfs":{"total":96636764160,"available":95930527744,"internally_reserved":0,"allocated":295702528,"data_stored":295433152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,3,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294969344,"data_stored":294965865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":241401856,"data_stored":241335162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274841600,"data_stored":274834049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:37.813 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:37.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15336", 'reported_seq': 15420, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.186415+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:37.186415+0000', 'last_peered': '2024-09-11T05:20:37.186415+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:37.186415+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19881984, 'num_objects': 4854, 'num_object_clones': 0, 'num_object_copies': 9708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15336, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1179}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:37.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:37.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14914", 'reported_seq': 17715, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.136894+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:37.136894+0000', 'last_peered': '2024-09-11T05:20:37.136894+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:37.136894+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19673112, 'num_objects': 4804, 'num_object_clones': 0, 'num_object_copies': 9608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4804, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13317, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1334, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2643}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:37.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:37.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15223", 'reported_seq': 15361, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.193227+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:37.193227+0000', 'last_peered': '2024-09-11T05:20:37.193227+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:37.193227+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20393984, 'num_objects': 4979, 'num_object_clones': 0, 'num_object_copies': 9958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4979, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12388, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:37.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:37.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15091", 'reported_seq': 15228, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.119383+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:37.119383+0000', 'last_peered': '2024-09-11T05:20:37.119383+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:37.119383+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19988480, 'num_objects': 4880, 'num_object_clones': 0, 'num_object_copies': 9760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4880, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13159, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1355}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14837", 'reported_seq': 15008, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.140620+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:37.140620+0000', 'last_peered': '2024-09-11T05:20:37.140620+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:37.140620+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19804160, 'num_objects': 4835, 'num_object_clones': 0, 'num_object_copies': 9670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11590, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1393}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15148", 'reported_seq': 15268, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.041306+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:37.041306+0000', 'last_peered': '2024-09-11T05:20:37.041306+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:37.041306+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20008960, 'num_objects': 4885, 'num_object_clones': 0, 'num_object_copies': 9770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13177, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1363}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15315", 'reported_seq': 15452, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.054057+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:37.054057+0000', 'last_peered': '2024-09-11T05:20:37.054057+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:37.054057+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19779584, 'num_objects': 4829, 'num_object_clones': 0, 'num_object_copies': 9658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12348, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1206}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'14851", 'reported_seq': 14990, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.096749+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:37.096749+0000', 'last_peered': '2024-09-11T05:20:37.096749+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:37.096749+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19673088, 'num_objects': 4803, 'num_object_clones': 0, 'num_object_copies': 9606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11717, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1308}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:37.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14952", 'reported_seq': 15093, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.112775+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:37.112775+0000', 'last_peered': '2024-09-11T05:20:37.112775+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:37.112775+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19857408, 'num_objects': 4848, 'num_object_clones': 0, 'num_object_copies': 9696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11710, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:37.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:37.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14862", 'reported_seq': 15011, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:37.038326+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:37.038326+0000', 'last_peered': '2024-09-11T05:20:37.038326+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:37.038326+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19366505, 'num_objects': 4729, 'num_object_clones': 0, 'num_object_copies': 9458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11869, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1214}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:37.816 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:38.776 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:38.776 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34095/154862 objects degraded (22.016%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":516,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77431,"data_bytes":317141652,"bytes_used":1884667904,"bytes_avail":288025624576,"bytes_total":289910292480,"degraded_objects":34095,"degraded_total":154862,"degraded_ratio":0.22016375870129534,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:38.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:38.830 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:39.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:39.519 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:39.612 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-11T05:20:37.574531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317141652,"num_objects":77431,"num_object_clones":0,"num_object_copies":154862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204268,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8418,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1840496,"kb_used_data":794492,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281275024,"statfs":{"total":289910292480,"available":288025624576,"internally_reserved":0,"allocated":813559808,"data_stored":812688470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1071041537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,8,12,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27295744,"num_objects":-6664,"num_object_clones":0,"num_object_copies":-13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002726"},"pg_stats":[{"pgid":"2.f","version":"47'15336","reported_seq":15420,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.186415+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:37.186415+0000","last_peered":"2024-09-11T05:20:37.186415+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:37.186415+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15133","reported_seq":15241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.153087+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:37.153087+0000","last_peered":"2024-09-11T05:20:37.153087+0000","last_clean":"2024-09-11T05:20:37.153087+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:37.153087+0000","last_undegraded":"2024-09-11T05:20:37.153087+0000","last_fullsized":"2024-09-11T05:20:37.153087+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13531,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14914","reported_seq":17715,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.136894+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:37.136894+0000","last_peered":"2024-09-11T05:20:37.136894+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:37.136894+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673112,"num_objects":4804,"num_object_clones":0,"num_object_copies":9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13317,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2643}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15205","reported_seq":15282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.122932+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:37.122932+0000","last_peered":"2024-09-11T05:20:37.122932+0000","last_clean":"2024-09-11T05:20:37.122932+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:37.122932+0000","last_undegraded":"2024-09-11T05:20:37.122932+0000","last_fullsized":"2024-09-11T05:20:37.122932+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19476480,"num_objects":4755,"num_object_clones":0,"num_object_copies":9510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15205,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15281","reported_seq":15416,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.184636+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:37.184636+0000","last_peered":"2024-09-11T05:20:37.184636+0000","last_clean":"2024-09-11T05:20:37.184636+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:37.184636+0000","last_undegraded":"2024-09-11T05:20:37.184636+0000","last_fullsized":"2024-09-11T05:20:37.184636+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12115,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15115","reported_seq":22466,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:34.674890+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:34.674890+0000","last_peered":"2024-09-11T05:20:34.674890+0000","last_clean":"2024-09-11T05:20:34.674890+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:34.674890+0000","last_undegraded":"2024-09-11T05:20:34.674890+0000","last_fullsized":"2024-09-11T05:20:34.674890+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20152320,"num_objects":4920,"num_object_clones":0,"num_object_copies":9840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15223","reported_seq":15361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.193227+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:37.193227+0000","last_peered":"2024-09-11T05:20:37.193227+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:37.193227+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15023","reported_seq":15152,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.181338+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:37.181338+0000","last_peered":"2024-09-11T05:20:37.181338+0000","last_clean":"2024-09-11T05:20:37.181338+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:37.181338+0000","last_undegraded":"2024-09-11T05:20:37.181338+0000","last_fullsized":"2024-09-11T05:20:37.181338+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19910656,"num_objects":4861,"num_object_clones":0,"num_object_copies":9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12326,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15091","reported_seq":15228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.119383+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:37.119383+0000","last_peered":"2024-09-11T05:20:37.119383+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:37.119383+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988480,"num_objects":4880,"num_object_clones":0,"num_object_copies":9760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13159,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1355}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14837","reported_seq":15008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.140620+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:37.140620+0000","last_peered":"2024-09-11T05:20:37.140620+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:37.140620+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19804160,"num_objects":4835,"num_object_clones":0,"num_object_copies":9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15148","reported_seq":15268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.041306+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:37.041306+0000","last_peered":"2024-09-11T05:20:37.041306+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:37.041306+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13177,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14916","reported_seq":22007,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.122521+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:37.122521+0000","last_peered":"2024-09-11T05:20:37.122521+0000","last_clean":"2024-09-11T05:20:37.122521+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:37.122521+0000","last_undegraded":"2024-09-11T05:20:37.122521+0000","last_fullsized":"2024-09-11T05:20:37.122521+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628032,"num_objects":4792,"num_object_clones":0,"num_object_copies":9584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12819,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15315","reported_seq":15452,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.054057+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:37.054057+0000","last_peered":"2024-09-11T05:20:37.054057+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:37.054057+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19779584,"num_objects":4829,"num_object_clones":0,"num_object_copies":9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14851","reported_seq":14990,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.096749+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:37.096749+0000","last_peered":"2024-09-11T05:20:37.096749+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:37.096749+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673088,"num_objects":4803,"num_object_clones":0,"num_object_copies":9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14952","reported_seq":15093,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.112775+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:37.112775+0000","last_peered":"2024-09-11T05:20:37.112775+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:37.112775+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11710,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14862","reported_seq":15011,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.038326+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:37.038326+0000","last_peered":"2024-09-11T05:20:37.038326+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:37.038326+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19366505,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11869,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1214}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317141633,"num_objects":77429,"num_object_clones":0,"num_object_copies":154858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204266,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8417,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811212800,"data_stored":811135076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567220,"kb_used_data":268960,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93804620,"statfs":{"total":96636764160,"available":96055930880,"internally_reserved":0,"allocated":275415040,"data_stored":275145669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583592,"kb_used_data":236760,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93788248,"statfs":{"total":96636764160,"available":96039165952,"internally_reserved":0,"allocated":242442240,"data_stored":242109649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215201,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689684,"kb_used_data":288772,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93682156,"statfs":{"total":96636764160,"available":95930527744,"internally_reserved":0,"allocated":295702528,"data_stored":295433152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,3,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294969344,"data_stored":294965865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":241401856,"data_stored":241335162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274841600,"data_stored":274834049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:42.615 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:43.515 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:43.516 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:43.594 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-11T05:20:41.575466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316568212,"num_objects":77291,"num_object_clones":0,"num_object_copies":154582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204408,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8418,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160645,"ondisk_log_size":160645,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1825628,"kb_used_data":787944,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281289892,"statfs":{"total":289910292480,"available":288040849408,"internally_reserved":0,"allocated":806854656,"data_stored":805983318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,10,12,16],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19288064,"num_objects":-4709,"num_object_clones":0,"num_object_copies":-9418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4709,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"2.f","version":"47'15336","reported_seq":15420,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.186415+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:37.186415+0000","last_peered":"2024-09-11T05:20:37.186415+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:37.186415+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15133","reported_seq":15241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.153087+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:37.153087+0000","last_peered":"2024-09-11T05:20:37.153087+0000","last_clean":"2024-09-11T05:20:37.153087+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:37.153087+0000","last_undegraded":"2024-09-11T05:20:37.153087+0000","last_fullsized":"2024-09-11T05:20:37.153087+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13531,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14914","reported_seq":17715,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.136894+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:37.136894+0000","last_peered":"2024-09-11T05:20:37.136894+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:37.136894+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673112,"num_objects":4804,"num_object_clones":0,"num_object_copies":9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13317,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2643}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15205","reported_seq":15282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.122932+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:37.122932+0000","last_peered":"2024-09-11T05:20:37.122932+0000","last_clean":"2024-09-11T05:20:37.122932+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:37.122932+0000","last_undegraded":"2024-09-11T05:20:37.122932+0000","last_fullsized":"2024-09-11T05:20:37.122932+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19476480,"num_objects":4755,"num_object_clones":0,"num_object_copies":9510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15205,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15281","reported_seq":15416,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.184636+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:37.184636+0000","last_peered":"2024-09-11T05:20:37.184636+0000","last_clean":"2024-09-11T05:20:37.184636+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:37.184636+0000","last_undegraded":"2024-09-11T05:20:37.184636+0000","last_fullsized":"2024-09-11T05:20:37.184636+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12115,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15255","reported_seq":22606,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:39.633806+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:39.633806+0000","last_peered":"2024-09-11T05:20:39.633806+0000","last_clean":"2024-09-11T05:20:39.633806+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:39.633806+0000","last_undegraded":"2024-09-11T05:20:39.633806+0000","last_fullsized":"2024-09-11T05:20:39.633806+0000","mapping_epoch":46,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15223","reported_seq":15361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.193227+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:37.193227+0000","last_peered":"2024-09-11T05:20:37.193227+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:37.193227+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15023","reported_seq":15152,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.181338+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:37.181338+0000","last_peered":"2024-09-11T05:20:37.181338+0000","last_clean":"2024-09-11T05:20:37.181338+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:37.181338+0000","last_undegraded":"2024-09-11T05:20:37.181338+0000","last_fullsized":"2024-09-11T05:20:37.181338+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19910656,"num_objects":4861,"num_object_clones":0,"num_object_copies":9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12326,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15091","reported_seq":15228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.119383+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:37.119383+0000","last_peered":"2024-09-11T05:20:37.119383+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:37.119383+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988480,"num_objects":4880,"num_object_clones":0,"num_object_copies":9760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13159,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1355}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14837","reported_seq":15008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.140620+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:37.140620+0000","last_peered":"2024-09-11T05:20:37.140620+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:37.140620+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19804160,"num_objects":4835,"num_object_clones":0,"num_object_copies":9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15148","reported_seq":15268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.041306+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:37.041306+0000","last_peered":"2024-09-11T05:20:37.041306+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:37.041306+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13177,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14916","reported_seq":22007,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:37.122521+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:37.122521+0000","last_peered":"2024-09-11T05:20:37.122521+0000","last_clean":"2024-09-11T05:20:37.122521+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:37.122521+0000","last_undegraded":"2024-09-11T05:20:37.122521+0000","last_fullsized":"2024-09-11T05:20:37.122521+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628032,"num_objects":4792,"num_object_clones":0,"num_object_copies":9584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12819,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15315","reported_seq":15452,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.054057+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:37.054057+0000","last_peered":"2024-09-11T05:20:37.054057+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:37.054057+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19779584,"num_objects":4829,"num_object_clones":0,"num_object_copies":9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14851","reported_seq":14990,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.096749+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:37.096749+0000","last_peered":"2024-09-11T05:20:37.096749+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:37.096749+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673088,"num_objects":4803,"num_object_clones":0,"num_object_copies":9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11717,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14952","reported_seq":15093,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.112775+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:37.112775+0000","last_peered":"2024-09-11T05:20:37.112775+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:37.112775+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":4848,"num_object_clones":0,"num_object_copies":9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11710,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14862","reported_seq":15011,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:37.038326+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:37.038326+0000","last_peered":"2024-09-11T05:20:37.038326+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:37.038326+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19366505,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11869,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1214}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":316568193,"num_objects":77289,"num_object_clones":0,"num_object_copies":154578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204406,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8417,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":804409344,"data_stored":804331620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160642,"ondisk_log_size":160642,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567220,"kb_used_data":268960,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93804620,"statfs":{"total":96636764160,"available":96055930880,"internally_reserved":0,"allocated":275415040,"data_stored":275145669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568724,"kb_used_data":230212,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93803116,"statfs":{"total":96636764160,"available":96054390784,"internally_reserved":0,"allocated":235737088,"data_stored":235404497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215201,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689684,"kb_used_data":288772,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93682156,"statfs":{"total":96636764160,"available":95930527744,"internally_reserved":0,"allocated":295702528,"data_stored":295433152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,3,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294969344,"data_stored":294965865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234598400,"data_stored":234531706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274841600,"data_stored":274834049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:43.595 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:44.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:44.383 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:44.453 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-11T05:20:43.575932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":307954324,"num_objects":75188,"num_object_clones":0,"num_object_copies":150376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206511,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8517,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1810192,"kb_used_data":776668,"kb_used_omap":12,"kb_used_meta":1033459,"kb_avail":281305328,"statfs":{"total":289910292480,"available":288056655872,"internally_reserved":0,"allocated":795308032,"data_stored":794436694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1058262017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,8,10,5,9,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27316224,"num_objects":-6669,"num_object_clones":0,"num_object_copies":-13338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002764"},"pg_stats":[{"pgid":"2.f","version":"47'15470","reported_seq":15554,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.174887+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:42.174887+0000","last_peered":"2024-09-11T05:20:42.174887+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:42.174887+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":4720,"num_object_clones":0,"num_object_copies":9440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15470,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1045}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15275","reported_seq":15383,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.199183+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:42.199183+0000","last_peered":"2024-09-11T05:20:42.199183+0000","last_clean":"2024-09-11T05:20:42.199183+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:42.199183+0000","last_undegraded":"2024-09-11T05:20:42.199183+0000","last_fullsized":"2024-09-11T05:20:42.199183+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18944000,"num_objects":4625,"num_object_clones":0,"num_object_copies":9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13673,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15052","reported_seq":18051,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.170498+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:42.170498+0000","last_peered":"2024-09-11T05:20:42.170498+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:42.170498+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19107864,"num_objects":4666,"num_object_clones":0,"num_object_copies":9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13455,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1433,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2604}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15338","reported_seq":15415,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.163514+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:42.163514+0000","last_peered":"2024-09-11T05:20:42.163514+0000","last_clean":"2024-09-11T05:20:42.163514+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:42.163514+0000","last_undegraded":"2024-09-11T05:20:42.163514+0000","last_fullsized":"2024-09-11T05:20:42.163514+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18931712,"num_objects":4622,"num_object_clones":0,"num_object_copies":9244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15338,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15429","reported_seq":15564,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.064433+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:42.064433+0000","last_peered":"2024-09-11T05:20:42.064433+0000","last_clean":"2024-09-11T05:20:42.064433+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:42.064433+0000","last_undegraded":"2024-09-11T05:20:42.064433+0000","last_fullsized":"2024-09-11T05:20:42.064433+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19415040,"num_objects":4740,"num_object_clones":0,"num_object_copies":9480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12263,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15255","reported_seq":22606,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:39.633806+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:39.633806+0000","last_peered":"2024-09-11T05:20:39.633806+0000","last_clean":"2024-09-11T05:20:39.633806+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:39.633806+0000","last_undegraded":"2024-09-11T05:20:39.633806+0000","last_fullsized":"2024-09-11T05:20:39.633806+0000","mapping_epoch":46,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15380","reported_seq":15518,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.153925+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:42.153925+0000","last_peered":"2024-09-11T05:20:42.153925+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:42.153925+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12545,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15164","reported_seq":15294,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.228543+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:42.228543+0000","last_peered":"2024-09-11T05:20:42.228543+0000","last_clean":"2024-09-11T05:20:42.228543+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:42.228543+0000","last_undegraded":"2024-09-11T05:20:42.228543+0000","last_fullsized":"2024-09-11T05:20:42.228543+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":4720,"num_object_clones":0,"num_object_copies":9440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12467,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15222","reported_seq":15358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.124300+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:42.124300+0000","last_peered":"2024-09-11T05:20:42.124300+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:42.124300+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19451904,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13290,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1224}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14978","reported_seq":15148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.169521+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:42.169521+0000","last_peered":"2024-09-11T05:20:42.169521+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:42.169521+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19226624,"num_objects":4694,"num_object_clones":0,"num_object_copies":9388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15280","reported_seq":15403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.114970+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:42.114970+0000","last_peered":"2024-09-11T05:20:42.114970+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:42.114970+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19468288,"num_objects":4753,"num_object_clones":0,"num_object_copies":9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13309,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1231}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15069","reported_seq":22159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.125670+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:42.125670+0000","last_peered":"2024-09-11T05:20:42.125670+0000","last_clean":"2024-09-11T05:20:42.125670+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:42.125670+0000","last_undegraded":"2024-09-11T05:20:42.125670+0000","last_fullsized":"2024-09-11T05:20:42.125670+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001344,"num_objects":4639,"num_object_clones":0,"num_object_copies":9278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12972,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15458","reported_seq":15595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.086263+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:42.086263+0000","last_peered":"2024-09-11T05:20:42.086263+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:42.086263+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193856,"num_objects":4686,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12491,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15001","reported_seq":15140,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.198921+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:42.198921+0000","last_peered":"2024-09-11T05:20:42.198921+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:42.198921+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19058688,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11867,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1158}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15105","reported_seq":15246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.222190+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:42.222190+0000","last_peered":"2024-09-11T05:20:42.222190+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:42.222190+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19230720,"num_objects":4695,"num_object_clones":0,"num_object_copies":9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708305+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708305+0000","last_peered":"2024-09-11T05:19:25.708305+0000","last_clean":"2024-09-11T05:19:25.708305+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708305+0000","last_undegraded":"2024-09-11T05:19:25.708305+0000","last_fullsized":"2024-09-11T05:19:25.708305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14969","reported_seq":15118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.138883+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:42.138883+0000","last_peered":"2024-09-11T05:20:42.138883+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:42.138883+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18928233,"num_objects":4622,"num_object_clones":0,"num_object_copies":9244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11976,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":307954305,"num_objects":75186,"num_object_clones":0,"num_object_copies":150372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75186,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206509,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8516,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":792838144,"data_stored":792760420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566956,"kb_used_data":264540,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93804884,"statfs":{"total":96636764160,"available":96056201216,"internally_reserved":0,"allocated":270888960,"data_stored":270619589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568724,"kb_used_data":230212,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93803116,"statfs":{"total":96636764160,"available":96054390784,"internally_reserved":0,"allocated":235737088,"data_stored":235404497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674512,"kb_used_data":281916,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93697328,"statfs":{"total":96636764160,"available":95946063872,"internally_reserved":0,"allocated":288681984,"data_stored":288412608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,5,2,2,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287997952,"data_stored":287994473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":234598400,"data_stored":234531706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270241792,"data_stored":270234241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:44.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:44.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15470", 'reported_seq': 15554, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.174887+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:42.174887+0000', 'last_peered': '2024-09-11T05:20:42.174887+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:42.174887+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19333120, 'num_objects': 4720, 'num_object_clones': 0, 'num_object_copies': 9440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15470, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1045}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:44.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:44.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15052", 'reported_seq': 18051, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.170498+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:42.170498+0000', 'last_peered': '2024-09-11T05:20:42.170498+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:42.170498+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19107864, 'num_objects': 4666, 'num_object_clones': 0, 'num_object_copies': 9332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2062, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4666, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13455, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1433, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2604}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15380", 'reported_seq': 15518, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.153925+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:42.153925+0000', 'last_peered': '2024-09-11T05:20:42.153925+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:42.153925+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19750912, 'num_objects': 4822, 'num_object_clones': 0, 'num_object_copies': 9644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12545, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15222", 'reported_seq': 15358, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.124300+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:42.124300+0000', 'last_peered': '2024-09-11T05:20:42.124300+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:42.124300+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19451904, 'num_objects': 4749, 'num_object_clones': 0, 'num_object_copies': 9498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13290, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1224}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14978", 'reported_seq': 15148, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.169521+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:42.169521+0000', 'last_peered': '2024-09-11T05:20:42.169521+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:42.169521+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19226624, 'num_objects': 4694, 'num_object_clones': 0, 'num_object_copies': 9388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11731, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1252}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15280", 'reported_seq': 15403, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.114970+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:42.114970+0000', 'last_peered': '2024-09-11T05:20:42.114970+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:42.114970+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19468288, 'num_objects': 4753, 'num_object_clones': 0, 'num_object_copies': 9506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13309, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1231}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:44.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15458", 'reported_seq': 15595, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.086263+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:42.086263+0000', 'last_peered': '2024-09-11T05:20:42.086263+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:42.086263+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19193856, 'num_objects': 4686, 'num_object_clones': 0, 'num_object_copies': 9372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12491, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:44.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:44.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15001", 'reported_seq': 15140, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.198921+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:42.198921+0000', 'last_peered': '2024-09-11T05:20:42.198921+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:42.198921+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19058688, 'num_objects': 4653, 'num_object_clones': 0, 'num_object_copies': 9306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4653, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11867, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1158}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:44.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:44.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15105", 'reported_seq': 15246, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.222190+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:42.222190+0000', 'last_peered': '2024-09-11T05:20:42.222190+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:42.222190+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19230720, 'num_objects': 4695, 'num_object_clones': 0, 'num_object_copies': 9390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11863, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1151}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:44.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:44.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14969", 'reported_seq': 15118, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:42.138883+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:42.138883+0000', 'last_peered': '2024-09-11T05:20:42.138883+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:42.138883+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18928233, 'num_objects': 4622, 'num_object_clones': 0, 'num_object_copies': 9244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4622, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11976, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:44.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:45.496 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33996/150376 objects degraded (22.607%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":523,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75188,"data_bytes":307954324,"bytes_used":1853636608,"bytes_avail":288056655872,"bytes_total":289910292480,"degraded_objects":33996,"degraded_total":150376,"degraded_ratio":0.22607330957067617,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":294,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:45.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:45.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:46.369 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:46.414 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-11T05:20:45.576367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":307397268,"num_objects":75052,"num_object_clones":0,"num_object_copies":150104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206647,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8517,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1807536,"kb_used_data":769848,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281307984,"statfs":{"total":289910292480,"available":288059375616,"internally_reserved":0,"allocated":788324352,"data_stored":787453014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,13,6,7,6,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19161088,"num_objects":-4678,"num_object_clones":0,"num_object_copies":-9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4678,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002750"},"pg_stats":[{"pgid":"2.f","version":"47'15470","reported_seq":15554,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.174887+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:42.174887+0000","last_peered":"2024-09-11T05:20:42.174887+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:42.174887+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":4720,"num_object_clones":0,"num_object_copies":9440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15470,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1045}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15275","reported_seq":15383,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.199183+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:42.199183+0000","last_peered":"2024-09-11T05:20:42.199183+0000","last_clean":"2024-09-11T05:20:42.199183+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:42.199183+0000","last_undegraded":"2024-09-11T05:20:42.199183+0000","last_fullsized":"2024-09-11T05:20:42.199183+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18944000,"num_objects":4625,"num_object_clones":0,"num_object_copies":9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13673,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15052","reported_seq":18051,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.170498+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:42.170498+0000","last_peered":"2024-09-11T05:20:42.170498+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:42.170498+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19107864,"num_objects":4666,"num_object_clones":0,"num_object_copies":9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13455,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1433,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2604}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15338","reported_seq":15415,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.163514+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:42.163514+0000","last_peered":"2024-09-11T05:20:42.163514+0000","last_clean":"2024-09-11T05:20:42.163514+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:42.163514+0000","last_undegraded":"2024-09-11T05:20:42.163514+0000","last_fullsized":"2024-09-11T05:20:42.163514+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18931712,"num_objects":4622,"num_object_clones":0,"num_object_copies":9244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15338,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15429","reported_seq":15564,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.064433+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:42.064433+0000","last_peered":"2024-09-11T05:20:42.064433+0000","last_clean":"2024-09-11T05:20:42.064433+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:42.064433+0000","last_undegraded":"2024-09-11T05:20:42.064433+0000","last_fullsized":"2024-09-11T05:20:42.064433+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19415040,"num_objects":4740,"num_object_clones":0,"num_object_copies":9480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12263,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15391","reported_seq":22742,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:44.525230+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:44.525230+0000","last_peered":"2024-09-11T05:20:44.525230+0000","last_clean":"2024-09-11T05:20:44.525230+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:44.525230+0000","last_undegraded":"2024-09-11T05:20:44.525230+0000","last_fullsized":"2024-09-11T05:20:44.525230+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11935,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15380","reported_seq":15518,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.153925+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:42.153925+0000","last_peered":"2024-09-11T05:20:42.153925+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:42.153925+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12545,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15164","reported_seq":15294,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.228543+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:42.228543+0000","last_peered":"2024-09-11T05:20:42.228543+0000","last_clean":"2024-09-11T05:20:42.228543+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:42.228543+0000","last_undegraded":"2024-09-11T05:20:42.228543+0000","last_fullsized":"2024-09-11T05:20:42.228543+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":4720,"num_object_clones":0,"num_object_copies":9440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12467,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15222","reported_seq":15358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.124300+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:42.124300+0000","last_peered":"2024-09-11T05:20:42.124300+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:42.124300+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19451904,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13290,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1224}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14978","reported_seq":15148,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.169521+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:42.169521+0000","last_peered":"2024-09-11T05:20:42.169521+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:42.169521+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19226624,"num_objects":4694,"num_object_clones":0,"num_object_copies":9388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15280","reported_seq":15403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.114970+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:42.114970+0000","last_peered":"2024-09-11T05:20:42.114970+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:42.114970+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19468288,"num_objects":4753,"num_object_clones":0,"num_object_copies":9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13309,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1231}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15069","reported_seq":22159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:42.125670+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:42.125670+0000","last_peered":"2024-09-11T05:20:42.125670+0000","last_clean":"2024-09-11T05:20:42.125670+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:42.125670+0000","last_undegraded":"2024-09-11T05:20:42.125670+0000","last_fullsized":"2024-09-11T05:20:42.125670+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001344,"num_objects":4639,"num_object_clones":0,"num_object_copies":9278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12972,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15458","reported_seq":15595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.086263+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:42.086263+0000","last_peered":"2024-09-11T05:20:42.086263+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:42.086263+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193856,"num_objects":4686,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12491,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15001","reported_seq":15140,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.198921+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:42.198921+0000","last_peered":"2024-09-11T05:20:42.198921+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:42.198921+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19058688,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11867,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1158}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15105","reported_seq":15246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.222190+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:42.222190+0000","last_peered":"2024-09-11T05:20:42.222190+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:42.222190+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19230720,"num_objects":4695,"num_object_clones":0,"num_object_copies":9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14969","reported_seq":15118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:42.138883+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:42.138883+0000","last_peered":"2024-09-11T05:20:42.138883+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:42.138883+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18928233,"num_objects":4622,"num_object_clones":0,"num_object_copies":9244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11976,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":307397249,"num_objects":75050,"num_object_clones":0,"num_object_copies":150100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206645,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8516,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":785772544,"data_stored":785694820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566956,"kb_used_data":264540,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93804884,"statfs":{"total":96636764160,"available":96056201216,"internally_reserved":0,"allocated":270888960,"data_stored":270619589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566068,"kb_used_data":223392,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93805772,"statfs":{"total":96636764160,"available":96057110528,"internally_reserved":0,"allocated":228753408,"data_stored":228420817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674512,"kb_used_data":281916,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93697328,"statfs":{"total":96636764160,"available":95946063872,"internally_reserved":0,"allocated":288681984,"data_stored":288412608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,5,2,2,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287997952,"data_stored":287994473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227532800,"data_stored":227466106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270241792,"data_stored":270234241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:49.418 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:50.228 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:50.230 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:50.295 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-11T05:20:49.577195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298824340,"num_objects":72959,"num_object_clones":0,"num_object_copies":145918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208740,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8621,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160977,"ondisk_log_size":160977,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804836,"kb_used_data":758832,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281310684,"statfs":{"total":289910292480,"available":288062140416,"internally_reserved":0,"allocated":777043968,"data_stored":776172630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1071041537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,12,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18317312,"num_objects":-4472,"num_object_clones":0,"num_object_copies":-8944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4472,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002663"},"pg_stats":[{"pgid":"2.f","version":"47'15599","reported_seq":15683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.255074+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:47.255074+0000","last_peered":"2024-09-11T05:20:47.255074+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:47.255074+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15599,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15409","reported_seq":15517,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.173244+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:47.173244+0000","last_peered":"2024-09-11T05:20:47.173244+0000","last_clean":"2024-09-11T05:20:47.173244+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:47.173244+0000","last_undegraded":"2024-09-11T05:20:47.173244+0000","last_fullsized":"2024-09-11T05:20:47.173244+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18395136,"num_objects":4491,"num_object_clones":0,"num_object_copies":8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15185","reported_seq":18392,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.204351+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:47.204351+0000","last_peered":"2024-09-11T05:20:47.204351+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:47.204351+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18563096,"num_objects":4533,"num_object_clones":0,"num_object_copies":9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13588,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2575}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15465","reported_seq":15542,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.191590+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:47.191590+0000","last_peered":"2024-09-11T05:20:47.191590+0000","last_clean":"2024-09-11T05:20:47.191590+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:47.191590+0000","last_undegraded":"2024-09-11T05:20:47.191590+0000","last_fullsized":"2024-09-11T05:20:47.191590+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15465,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15573","reported_seq":15708,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.170361+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:47.170361+0000","last_peered":"2024-09-11T05:20:47.170361+0000","last_clean":"2024-09-11T05:20:47.170361+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:47.170361+0000","last_undegraded":"2024-09-11T05:20:47.170361+0000","last_fullsized":"2024-09-11T05:20:47.170361+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18825216,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12407,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15391","reported_seq":22742,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:44.525230+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:44.525230+0000","last_peered":"2024-09-11T05:20:44.525230+0000","last_clean":"2024-09-11T05:20:44.525230+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:44.525230+0000","last_undegraded":"2024-09-11T05:20:44.525230+0000","last_fullsized":"2024-09-11T05:20:44.525230+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11935,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15539","reported_seq":15676,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.206537+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:47.206537+0000","last_peered":"2024-09-11T05:20:47.206537+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:47.206537+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099648,"num_objects":4663,"num_object_clones":0,"num_object_copies":9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12704,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15294","reported_seq":15424,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.151326+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:47.151326+0000","last_peered":"2024-09-11T05:20:47.151326+0000","last_clean":"2024-09-11T05:20:47.151326+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:47.151326+0000","last_undegraded":"2024-09-11T05:20:47.151326+0000","last_fullsized":"2024-09-11T05:20:47.151326+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15378","reported_seq":15515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.144130+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:47.144130+0000","last_peered":"2024-09-11T05:20:47.144130+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:47.144130+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13446,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15135","reported_seq":15306,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.226497+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:47.226497+0000","last_peered":"2024-09-11T05:20:47.226497+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:47.226497+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18583552,"num_objects":4537,"num_object_clones":0,"num_object_copies":9074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11888,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1095}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15436","reported_seq":15559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.178426+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:47.178426+0000","last_peered":"2024-09-11T05:20:47.178426+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:47.178426+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18829312,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13465,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1075}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15194","reported_seq":22285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.186734+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:47.186734+0000","last_peered":"2024-09-11T05:20:47.186734+0000","last_clean":"2024-09-11T05:20:47.186734+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:47.186734+0000","last_undegraded":"2024-09-11T05:20:47.186734+0000","last_fullsized":"2024-09-11T05:20:47.186734+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13097,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15587","reported_seq":15724,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.256304+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:47.256304+0000","last_peered":"2024-09-11T05:20:47.256304+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:47.256304+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12620,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15142","reported_seq":15281,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.244098+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:47.244098+0000","last_peered":"2024-09-11T05:20:47.244098+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:47.244098+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12008,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15239","reported_seq":15379,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.207586+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:47.207586+0000","last_peered":"2024-09-11T05:20:47.207586+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:47.207586+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18681856,"num_objects":4561,"num_object_clones":0,"num_object_copies":9122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15108","reported_seq":15257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.217891+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:47.217891+0000","last_peered":"2024-09-11T05:20:47.217891+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:47.217891+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18358889,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12115,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298824321,"num_objects":72957,"num_object_clones":0,"num_object_copies":145914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72957,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208738,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8620,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":774463488,"data_stored":774385764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160974,"ondisk_log_size":160974,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566700,"kb_used_data":260124,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93805140,"statfs":{"total":96636764160,"available":96056463360,"internally_reserved":0,"allocated":266366976,"data_stored":266097605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566068,"kb_used_data":223392,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93805772,"statfs":{"total":96636764160,"available":96057110528,"internally_reserved":0,"allocated":228753408,"data_stored":228420817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215203,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672068,"kb_used_data":275316,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93699772,"statfs":{"total":96636764160,"available":95948566528,"internally_reserved":0,"allocated":281923584,"data_stored":281654208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":281219072,"data_stored":281215593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227532800,"data_stored":227466106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265711616,"data_stored":265704065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:50.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:51.000 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:51.001 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:51.097 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-11T05:20:49.577195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298824340,"num_objects":72959,"num_object_clones":0,"num_object_copies":145918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208740,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8621,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160977,"ondisk_log_size":160977,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804836,"kb_used_data":758832,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281310684,"statfs":{"total":289910292480,"available":288062140416,"internally_reserved":0,"allocated":777043968,"data_stored":776172630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1071041537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,12,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18317312,"num_objects":-4472,"num_object_clones":0,"num_object_copies":-8944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4472,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002663"},"pg_stats":[{"pgid":"2.f","version":"47'15599","reported_seq":15683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.255074+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:47.255074+0000","last_peered":"2024-09-11T05:20:47.255074+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:47.255074+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15599,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15409","reported_seq":15517,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.173244+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:47.173244+0000","last_peered":"2024-09-11T05:20:47.173244+0000","last_clean":"2024-09-11T05:20:47.173244+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:47.173244+0000","last_undegraded":"2024-09-11T05:20:47.173244+0000","last_fullsized":"2024-09-11T05:20:47.173244+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18395136,"num_objects":4491,"num_object_clones":0,"num_object_copies":8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15185","reported_seq":18392,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.204351+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:47.204351+0000","last_peered":"2024-09-11T05:20:47.204351+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:47.204351+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18563096,"num_objects":4533,"num_object_clones":0,"num_object_copies":9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13588,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2575}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15465","reported_seq":15542,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.191590+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:47.191590+0000","last_peered":"2024-09-11T05:20:47.191590+0000","last_clean":"2024-09-11T05:20:47.191590+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:47.191590+0000","last_undegraded":"2024-09-11T05:20:47.191590+0000","last_fullsized":"2024-09-11T05:20:47.191590+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15465,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15573","reported_seq":15708,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.170361+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:47.170361+0000","last_peered":"2024-09-11T05:20:47.170361+0000","last_clean":"2024-09-11T05:20:47.170361+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:47.170361+0000","last_undegraded":"2024-09-11T05:20:47.170361+0000","last_fullsized":"2024-09-11T05:20:47.170361+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18825216,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12407,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15391","reported_seq":22742,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:44.525230+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:44.525230+0000","last_peered":"2024-09-11T05:20:44.525230+0000","last_clean":"2024-09-11T05:20:44.525230+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:44.525230+0000","last_undegraded":"2024-09-11T05:20:44.525230+0000","last_fullsized":"2024-09-11T05:20:44.525230+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11935,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15539","reported_seq":15676,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.206537+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:47.206537+0000","last_peered":"2024-09-11T05:20:47.206537+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:47.206537+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099648,"num_objects":4663,"num_object_clones":0,"num_object_copies":9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12704,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15294","reported_seq":15424,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.151326+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:47.151326+0000","last_peered":"2024-09-11T05:20:47.151326+0000","last_clean":"2024-09-11T05:20:47.151326+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:47.151326+0000","last_undegraded":"2024-09-11T05:20:47.151326+0000","last_fullsized":"2024-09-11T05:20:47.151326+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15378","reported_seq":15515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.144130+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:47.144130+0000","last_peered":"2024-09-11T05:20:47.144130+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:47.144130+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13446,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15135","reported_seq":15306,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.226497+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:47.226497+0000","last_peered":"2024-09-11T05:20:47.226497+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:47.226497+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18583552,"num_objects":4537,"num_object_clones":0,"num_object_copies":9074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11888,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1095}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15436","reported_seq":15559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.178426+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:47.178426+0000","last_peered":"2024-09-11T05:20:47.178426+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:47.178426+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18829312,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13465,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1075}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15194","reported_seq":22285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.186734+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:47.186734+0000","last_peered":"2024-09-11T05:20:47.186734+0000","last_clean":"2024-09-11T05:20:47.186734+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:47.186734+0000","last_undegraded":"2024-09-11T05:20:47.186734+0000","last_fullsized":"2024-09-11T05:20:47.186734+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13097,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15587","reported_seq":15724,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.256304+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:47.256304+0000","last_peered":"2024-09-11T05:20:47.256304+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:47.256304+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12620,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15142","reported_seq":15281,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.244098+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:47.244098+0000","last_peered":"2024-09-11T05:20:47.244098+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:47.244098+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12008,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15239","reported_seq":15379,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.207586+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:47.207586+0000","last_peered":"2024-09-11T05:20:47.207586+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:47.207586+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18681856,"num_objects":4561,"num_object_clones":0,"num_object_copies":9122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15108","reported_seq":15257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.217891+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:47.217891+0000","last_peered":"2024-09-11T05:20:47.217891+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:47.217891+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18358889,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12115,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298824321,"num_objects":72957,"num_object_clones":0,"num_object_copies":145914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72957,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208738,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8620,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":774463488,"data_stored":774385764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160974,"ondisk_log_size":160974,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566700,"kb_used_data":260124,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93805140,"statfs":{"total":96636764160,"available":96056463360,"internally_reserved":0,"allocated":266366976,"data_stored":266097605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566068,"kb_used_data":223392,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93805772,"statfs":{"total":96636764160,"available":96057110528,"internally_reserved":0,"allocated":228753408,"data_stored":228420817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,7,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215203,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672068,"kb_used_data":275316,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93699772,"statfs":{"total":96636764160,"available":95948566528,"internally_reserved":0,"allocated":281923584,"data_stored":281654208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":281219072,"data_stored":281215593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":227532800,"data_stored":227466106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265711616,"data_stored":265704065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:51.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:51.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15599", 'reported_seq': 15683, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.255074+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:47.255074+0000', 'last_peered': '2024-09-11T05:20:47.255074+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:47.255074+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18804736, 'num_objects': 4591, 'num_object_clones': 0, 'num_object_copies': 9182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15599, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 916}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:51.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:51.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15185", 'reported_seq': 18392, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.204351+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:47.204351+0000', 'last_peered': '2024-09-11T05:20:47.204351+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:47.204351+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18563096, 'num_objects': 4533, 'num_object_clones': 0, 'num_object_copies': 9066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1958, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4533, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13588, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1537, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2575}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:51.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:51.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15539", 'reported_seq': 15676, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.206537+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:47.206537+0000', 'last_peered': '2024-09-11T05:20:47.206537+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:47.206537+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19099648, 'num_objects': 4663, 'num_object_clones': 0, 'num_object_copies': 9326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12704, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1087}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:51.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:51.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15378", 'reported_seq': 15515, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.144130+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:47.144130+0000', 'last_peered': '2024-09-11T05:20:47.144130+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:47.144130+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18812928, 'num_objects': 4593, 'num_object_clones': 0, 'num_object_copies': 9186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13446, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:51.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:51.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15135", 'reported_seq': 15306, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.226497+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:47.226497+0000', 'last_peered': '2024-09-11T05:20:47.226497+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:47.226497+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18583552, 'num_objects': 4537, 'num_object_clones': 0, 'num_object_copies': 9074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11888, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1095}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:51.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:51.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15436", 'reported_seq': 15559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.178426+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:47.178426+0000', 'last_peered': '2024-09-11T05:20:47.178426+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:47.178426+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18829312, 'num_objects': 4597, 'num_object_clones': 0, 'num_object_copies': 9194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13465, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1075}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:51.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:51.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15587", 'reported_seq': 15724, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.256304+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:47.256304+0000', 'last_peered': '2024-09-11T05:20:47.256304+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:47.256304+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18665472, 'num_objects': 4557, 'num_object_clones': 0, 'num_object_copies': 9114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4557, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12620, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:51.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:51.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15142", 'reported_seq': 15281, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.244098+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:47.244098+0000', 'last_peered': '2024-09-11T05:20:47.244098+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:47.244098+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18481152, 'num_objects': 4512, 'num_object_clones': 0, 'num_object_copies': 9024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12008, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1017}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:51.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:51.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15239", 'reported_seq': 15379, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.207586+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:47.207586+0000', 'last_peered': '2024-09-11T05:20:47.207586+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:47.207586+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18681856, 'num_objects': 4561, 'num_object_clones': 0, 'num_object_copies': 9122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11997, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1017}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:51.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:51.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15108", 'reported_seq': 15257, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:47.217891+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:47.217891+0000', 'last_peered': '2024-09-11T05:20:47.217891+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:47.217891+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18358889, 'num_objects': 4483, 'num_object_clones': 0, 'num_object_copies': 8966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12115, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:51.103 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:51.948 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33892/145640 objects degraded (23.271%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":529,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72820,"data_bytes":298254996,"bytes_used":1832685568,"bytes_avail":288077606912,"bytes_total":289910292480,"degraded_objects":33892,"degraded_total":145640,"degraded_ratio":0.23271079373798409,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:52.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:52.002 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:52.812 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:52.859 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-11T05:20:51.577673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298254996,"num_objects":72820,"num_object_clones":0,"num_object_copies":145640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208879,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8621,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1789732,"kb_used_data":752052,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281325788,"statfs":{"total":289910292480,"available":288077606912,"internally_reserved":0,"allocated":770101248,"data_stored":769229910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,10,7,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18886656,"num_objects":-4611,"num_object_clones":0,"num_object_copies":-9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002652"},"pg_stats":[{"pgid":"2.f","version":"47'15599","reported_seq":15683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.255074+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:47.255074+0000","last_peered":"2024-09-11T05:20:47.255074+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:47.255074+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15599,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15409","reported_seq":15517,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.173244+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:47.173244+0000","last_peered":"2024-09-11T05:20:47.173244+0000","last_clean":"2024-09-11T05:20:47.173244+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:47.173244+0000","last_undegraded":"2024-09-11T05:20:47.173244+0000","last_fullsized":"2024-09-11T05:20:47.173244+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18395136,"num_objects":4491,"num_object_clones":0,"num_object_copies":8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15185","reported_seq":18392,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.204351+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:47.204351+0000","last_peered":"2024-09-11T05:20:47.204351+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:47.204351+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18563096,"num_objects":4533,"num_object_clones":0,"num_object_copies":9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13588,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2575}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15465","reported_seq":15542,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.191590+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:47.191590+0000","last_peered":"2024-09-11T05:20:47.191590+0000","last_clean":"2024-09-11T05:20:47.191590+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:47.191590+0000","last_undegraded":"2024-09-11T05:20:47.191590+0000","last_fullsized":"2024-09-11T05:20:47.191590+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15465,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15573","reported_seq":15708,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.170361+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:47.170361+0000","last_peered":"2024-09-11T05:20:47.170361+0000","last_clean":"2024-09-11T05:20:47.170361+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:47.170361+0000","last_undegraded":"2024-09-11T05:20:47.170361+0000","last_fullsized":"2024-09-11T05:20:47.170361+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18825216,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12407,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15530","reported_seq":22881,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:49.779925+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:49.779925+0000","last_peered":"2024-09-11T05:20:49.779925+0000","last_clean":"2024-09-11T05:20:49.779925+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:49.779925+0000","last_undegraded":"2024-09-11T05:20:49.779925+0000","last_fullsized":"2024-09-11T05:20:49.779925+0000","mapping_epoch":46,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12074,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15539","reported_seq":15676,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.206537+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:47.206537+0000","last_peered":"2024-09-11T05:20:47.206537+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:47.206537+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099648,"num_objects":4663,"num_object_clones":0,"num_object_copies":9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12704,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15294","reported_seq":15424,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.151326+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:47.151326+0000","last_peered":"2024-09-11T05:20:47.151326+0000","last_clean":"2024-09-11T05:20:47.151326+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:47.151326+0000","last_undegraded":"2024-09-11T05:20:47.151326+0000","last_fullsized":"2024-09-11T05:20:47.151326+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15378","reported_seq":15515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.144130+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:47.144130+0000","last_peered":"2024-09-11T05:20:47.144130+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:47.144130+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18812928,"num_objects":4593,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13446,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15135","reported_seq":15306,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.226497+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:47.226497+0000","last_peered":"2024-09-11T05:20:47.226497+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:47.226497+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18583552,"num_objects":4537,"num_object_clones":0,"num_object_copies":9074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11888,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1095}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15436","reported_seq":15559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.178426+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:47.178426+0000","last_peered":"2024-09-11T05:20:47.178426+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:47.178426+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18829312,"num_objects":4597,"num_object_clones":0,"num_object_copies":9194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13465,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1075}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15194","reported_seq":22285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:47.186734+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:47.186734+0000","last_peered":"2024-09-11T05:20:47.186734+0000","last_clean":"2024-09-11T05:20:47.186734+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:47.186734+0000","last_undegraded":"2024-09-11T05:20:47.186734+0000","last_fullsized":"2024-09-11T05:20:47.186734+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13097,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15587","reported_seq":15724,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.256304+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:47.256304+0000","last_peered":"2024-09-11T05:20:47.256304+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:47.256304+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12620,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15142","reported_seq":15281,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.244098+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:47.244098+0000","last_peered":"2024-09-11T05:20:47.244098+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:47.244098+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12008,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15239","reported_seq":15379,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.207586+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:47.207586+0000","last_peered":"2024-09-11T05:20:47.207586+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:47.207586+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18681856,"num_objects":4561,"num_object_clones":0,"num_object_copies":9122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15108","reported_seq":15257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:47.217891+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:47.217891+0000","last_peered":"2024-09-11T05:20:47.217891+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:47.217891+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18358889,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12115,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298254977,"num_objects":72818,"num_object_clones":0,"num_object_copies":145636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72818,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208877,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8620,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":767385600,"data_stored":767307876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566700,"kb_used_data":260124,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93805140,"statfs":{"total":96636764160,"available":96056463360,"internally_reserved":0,"allocated":266366976,"data_stored":266097605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550964,"kb_used_data":216612,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93820876,"statfs":{"total":96636764160,"available":96072577024,"internally_reserved":0,"allocated":221810688,"data_stored":221478097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215203,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672068,"kb_used_data":275316,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93699772,"statfs":{"total":96636764160,"available":95948566528,"internally_reserved":0,"allocated":281923584,"data_stored":281654208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":281219072,"data_stored":281215593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":220454912,"data_stored":220388218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265711616,"data_stored":265704065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:55.861 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:56.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:56.648 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:56.683 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-11T05:20:55.579776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289104532,"num_objects":70586,"num_object_clones":0,"num_object_copies":141172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211113,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8729,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1788528,"kb_used_data":734200,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281326992,"statfs":{"total":289910292480,"available":288078839808,"internally_reserved":0,"allocated":751820800,"data_stored":750949462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1079561217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,5,5,12,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18849792,"num_objects":-4602,"num_object_clones":0,"num_object_copies":-9204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4602,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003843"},"pg_stats":[{"pgid":"2.f","version":"47'15741","reported_seq":15825,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.266616+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:52.266616+0000","last_peered":"2024-09-11T05:20:52.266616+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:52.266616+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15741,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":774}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15544","reported_seq":15652,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.212507+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:52.212507+0000","last_peered":"2024-09-11T05:20:52.212507+0000","last_clean":"2024-09-11T05:20:52.212507+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:52.212507+0000","last_undegraded":"2024-09-11T05:20:52.212507+0000","last_fullsized":"2024-09-11T05:20:52.212507+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17842176,"num_objects":4356,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13942,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15337","reported_seq":18760,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.244369+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:52.244369+0000","last_peered":"2024-09-11T05:20:52.244369+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:52.244369+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940504,"num_objects":4381,"num_object_clones":0,"num_object_copies":8762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13740,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15611","reported_seq":15688,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.192550+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:52.192550+0000","last_peered":"2024-09-11T05:20:52.192550+0000","last_clean":"2024-09-11T05:20:52.192550+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:52.192550+0000","last_undegraded":"2024-09-11T05:20:52.192550+0000","last_fullsized":"2024-09-11T05:20:52.192550+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17813504,"num_objects":4349,"num_object_clones":0,"num_object_copies":8698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15611,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15712","reported_seq":15847,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.252922+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:52.252922+0000","last_peered":"2024-09-11T05:20:52.252922+0000","last_clean":"2024-09-11T05:20:52.252922+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:52.252922+0000","last_undegraded":"2024-09-11T05:20:52.252922+0000","last_fullsized":"2024-09-11T05:20:52.252922+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18255872,"num_objects":4457,"num_object_clones":0,"num_object_copies":8914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12546,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15643","reported_seq":22994,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:54.754795+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:54.754795+0000","last_peered":"2024-09-11T05:20:54.754795+0000","last_clean":"2024-09-11T05:20:54.754795+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:54.754795+0000","last_undegraded":"2024-09-11T05:20:54.754795+0000","last_fullsized":"2024-09-11T05:20:54.754795+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17989632,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12187,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15697","reported_seq":15830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.254386+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:52.254386+0000","last_peered":"2024-09-11T05:20:52.254386+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:52.254386+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12862,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":929}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15434","reported_seq":15564,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.192394+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:52.192394+0000","last_peered":"2024-09-11T05:20:52.192394+0000","last_clean":"2024-09-11T05:20:52.192394+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:52.192394+0000","last_undegraded":"2024-09-11T05:20:52.192394+0000","last_fullsized":"2024-09-11T05:20:52.192394+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12737,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15524","reported_seq":15661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.184889+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:52.184889+0000","last_peered":"2024-09-11T05:20:52.184889+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:52.184889+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13592,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15271","reported_seq":15442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.274905+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:52.274905+0000","last_peered":"2024-09-11T05:20:52.274905+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:52.274905+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18026496,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12024,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":959}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15597","reported_seq":15720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.109308+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:52.109308+0000","last_peered":"2024-09-11T05:20:52.109308+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:52.109308+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18169856,"num_objects":4436,"num_object_clones":0,"num_object_copies":8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13626,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":914}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15330","reported_seq":22421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.118854+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:52.118854+0000","last_peered":"2024-09-11T05:20:52.118854+0000","last_clean":"2024-09-11T05:20:52.118854+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:52.118854+0000","last_undegraded":"2024-09-11T05:20:52.118854+0000","last_fullsized":"2024-09-11T05:20:52.118854+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17932288,"num_objects":4378,"num_object_clones":0,"num_object_copies":8756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15743","reported_seq":15879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.267872+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:52.267872+0000","last_peered":"2024-09-11T05:20:52.267872+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:52.267872+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18026496,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12776,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15269","reported_seq":15405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.294584+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:52.294584+0000","last_peered":"2024-09-11T05:20:52.294584+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:52.294584+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12135,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":890}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15363","reported_seq":15504,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.200191+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:52.200191+0000","last_peered":"2024-09-11T05:20:52.200191+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:52.200191+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18173952,"num_objects":4437,"num_object_clones":0,"num_object_copies":8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12121,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":893}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15231","reported_seq":15380,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.188994+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:52.188994+0000","last_peered":"2024-09-11T05:20:52.188994+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:52.188994+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17855081,"num_objects":4360,"num_object_clones":0,"num_object_copies":8720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12238,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289104513,"num_objects":70584,"num_object_clones":0,"num_object_copies":141168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70584,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211111,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8728,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":749060096,"data_stored":748982372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566276,"kb_used_data":255536,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93805564,"statfs":{"total":96636764160,"available":96056897536,"internally_reserved":0,"allocated":261668864,"data_stored":261399493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552648,"kb_used_data":209976,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93819192,"statfs":{"total":96636764160,"available":96070852608,"internally_reserved":0,"allocated":215015424,"data_stored":214682833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215204,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669604,"kb_used_data":268688,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93702236,"statfs":{"total":96636764160,"available":95951089664,"internally_reserved":0,"allocated":275136512,"data_stored":274867136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274501632,"data_stored":274498153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213561344,"data_stored":213494650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260997120,"data_stored":260989569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:56.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:57.543 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:57.544 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:57.575 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-11T05:20:55.579776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289104532,"num_objects":70586,"num_object_clones":0,"num_object_copies":141172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211113,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8729,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1788528,"kb_used_data":734200,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281326992,"statfs":{"total":289910292480,"available":288078839808,"internally_reserved":0,"allocated":751820800,"data_stored":750949462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1079561217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,5,5,12,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18849792,"num_objects":-4602,"num_object_clones":0,"num_object_copies":-9204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4602,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003843"},"pg_stats":[{"pgid":"2.f","version":"47'15741","reported_seq":15825,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.266616+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:52.266616+0000","last_peered":"2024-09-11T05:20:52.266616+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:52.266616+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15741,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":774}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15544","reported_seq":15652,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.212507+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:52.212507+0000","last_peered":"2024-09-11T05:20:52.212507+0000","last_clean":"2024-09-11T05:20:52.212507+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:52.212507+0000","last_undegraded":"2024-09-11T05:20:52.212507+0000","last_fullsized":"2024-09-11T05:20:52.212507+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17842176,"num_objects":4356,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13942,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15337","reported_seq":18760,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.244369+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:52.244369+0000","last_peered":"2024-09-11T05:20:52.244369+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:52.244369+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940504,"num_objects":4381,"num_object_clones":0,"num_object_copies":8762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13740,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15611","reported_seq":15688,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.192550+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:52.192550+0000","last_peered":"2024-09-11T05:20:52.192550+0000","last_clean":"2024-09-11T05:20:52.192550+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:52.192550+0000","last_undegraded":"2024-09-11T05:20:52.192550+0000","last_fullsized":"2024-09-11T05:20:52.192550+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17813504,"num_objects":4349,"num_object_clones":0,"num_object_copies":8698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15611,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15712","reported_seq":15847,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.252922+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:52.252922+0000","last_peered":"2024-09-11T05:20:52.252922+0000","last_clean":"2024-09-11T05:20:52.252922+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:52.252922+0000","last_undegraded":"2024-09-11T05:20:52.252922+0000","last_fullsized":"2024-09-11T05:20:52.252922+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18255872,"num_objects":4457,"num_object_clones":0,"num_object_copies":8914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12546,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15643","reported_seq":22994,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:54.754795+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:54.754795+0000","last_peered":"2024-09-11T05:20:54.754795+0000","last_clean":"2024-09-11T05:20:54.754795+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:54.754795+0000","last_undegraded":"2024-09-11T05:20:54.754795+0000","last_fullsized":"2024-09-11T05:20:54.754795+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17989632,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12187,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15697","reported_seq":15830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.254386+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:52.254386+0000","last_peered":"2024-09-11T05:20:52.254386+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:52.254386+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12862,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":929}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15434","reported_seq":15564,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.192394+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:52.192394+0000","last_peered":"2024-09-11T05:20:52.192394+0000","last_clean":"2024-09-11T05:20:52.192394+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:52.192394+0000","last_undegraded":"2024-09-11T05:20:52.192394+0000","last_fullsized":"2024-09-11T05:20:52.192394+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12737,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15524","reported_seq":15661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.184889+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:52.184889+0000","last_peered":"2024-09-11T05:20:52.184889+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:52.184889+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13592,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15271","reported_seq":15442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.274905+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:52.274905+0000","last_peered":"2024-09-11T05:20:52.274905+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:52.274905+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18026496,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12024,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":959}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15597","reported_seq":15720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.109308+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:52.109308+0000","last_peered":"2024-09-11T05:20:52.109308+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:52.109308+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18169856,"num_objects":4436,"num_object_clones":0,"num_object_copies":8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13626,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":914}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15330","reported_seq":22421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:52.118854+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:52.118854+0000","last_peered":"2024-09-11T05:20:52.118854+0000","last_clean":"2024-09-11T05:20:52.118854+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:52.118854+0000","last_undegraded":"2024-09-11T05:20:52.118854+0000","last_fullsized":"2024-09-11T05:20:52.118854+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17932288,"num_objects":4378,"num_object_clones":0,"num_object_copies":8756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15743","reported_seq":15879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.267872+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:52.267872+0000","last_peered":"2024-09-11T05:20:52.267872+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:52.267872+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18026496,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12776,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15269","reported_seq":15405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.294584+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:52.294584+0000","last_peered":"2024-09-11T05:20:52.294584+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:52.294584+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12135,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":890}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15363","reported_seq":15504,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.200191+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:52.200191+0000","last_peered":"2024-09-11T05:20:52.200191+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:52.200191+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18173952,"num_objects":4437,"num_object_clones":0,"num_object_copies":8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12121,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":893}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15231","reported_seq":15380,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:52.188994+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:52.188994+0000","last_peered":"2024-09-11T05:20:52.188994+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:52.188994+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17855081,"num_objects":4360,"num_object_clones":0,"num_object_copies":8720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12238,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289104513,"num_objects":70584,"num_object_clones":0,"num_object_copies":141168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70584,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211111,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8728,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":749060096,"data_stored":748982372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566276,"kb_used_data":255536,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93805564,"statfs":{"total":96636764160,"available":96056897536,"internally_reserved":0,"allocated":261668864,"data_stored":261399493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552648,"kb_used_data":209976,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93819192,"statfs":{"total":96636764160,"available":96070852608,"internally_reserved":0,"allocated":215015424,"data_stored":214682833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215204,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669604,"kb_used_data":268688,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93702236,"statfs":{"total":96636764160,"available":95951089664,"internally_reserved":0,"allocated":275136512,"data_stored":274867136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274501632,"data_stored":274498153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213561344,"data_stored":213494650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260997120,"data_stored":260989569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:20:57.577 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:20:57.577 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15741", 'reported_seq': 15825, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.266616+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:20:52.266616+0000', 'last_peered': '2024-09-11T05:20:52.266616+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:20:52.266616+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18223104, 'num_objects': 4449, 'num_object_clones': 0, 'num_object_copies': 8898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15741, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 774}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:57.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:20:57.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15337", 'reported_seq': 18760, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.244369+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:20:52.244369+0000', 'last_peered': '2024-09-11T05:20:52.244369+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:20:52.244369+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17940504, 'num_objects': 4381, 'num_object_clones': 0, 'num_object_copies': 8762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1850, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4381, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13740, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1645, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2531}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:57.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:20:57.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15697", 'reported_seq': 15830, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.254386+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:20:52.254386+0000', 'last_peered': '2024-09-11T05:20:52.254386+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:20:52.254386+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18452480, 'num_objects': 4505, 'num_object_clones': 0, 'num_object_copies': 9010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12862, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 929}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:57.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:20:57.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15524", 'reported_seq': 15661, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.184889+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:20:52.184889+0000', 'last_peered': '2024-09-11T05:20:52.184889+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:20:52.184889+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18214912, 'num_objects': 4447, 'num_object_clones': 0, 'num_object_copies': 8894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13592, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:57.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:20:57.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15271", 'reported_seq': 15442, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.274905+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:20:52.274905+0000', 'last_peered': '2024-09-11T05:20:52.274905+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:20:52.274905+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18026496, 'num_objects': 4401, 'num_object_clones': 0, 'num_object_copies': 8802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12024, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 959}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:57.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:20:57.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15597", 'reported_seq': 15720, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.109308+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:20:52.109308+0000', 'last_peered': '2024-09-11T05:20:52.109308+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:20:52.109308+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18169856, 'num_objects': 4436, 'num_object_clones': 0, 'num_object_copies': 8872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13626, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 914}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:20:57.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:20:57.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15743", 'reported_seq': 15879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.267872+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:20:52.267872+0000', 'last_peered': '2024-09-11T05:20:52.267872+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:20:52.267872+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18026496, 'num_objects': 4401, 'num_object_clones': 0, 'num_object_copies': 8802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12776, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:57.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:20:57.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15269", 'reported_seq': 15405, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.294584+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:20:52.294584+0000', 'last_peered': '2024-09-11T05:20:52.294584+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:20:52.294584+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17960960, 'num_objects': 4385, 'num_object_clones': 0, 'num_object_copies': 8770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12135, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 890}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:57.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:20:57.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15363", 'reported_seq': 15504, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.200191+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:20:52.200191+0000', 'last_peered': '2024-09-11T05:20:52.200191+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:20:52.200191+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18173952, 'num_objects': 4437, 'num_object_clones': 0, 'num_object_copies': 8874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12121, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 893}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:57.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:20:57.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15231", 'reported_seq': 15380, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:20:52.188994+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:20:52.188994+0000', 'last_peered': '2024-09-11T05:20:52.188994+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:20:52.188994+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17855081, 'num_objects': 4360, 'num_object_clones': 0, 'num_object_copies': 8720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4360, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12238, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 845}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:20:57.580 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:20:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:58.611 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33693/136858 objects degraded (24.619%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":536,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68429,"data_bytes":280269460,"bytes_used":1798569984,"bytes_avail":288111722496,"bytes_total":289910292480,"degraded_objects":33693,"degraded_total":136858,"degraded_ratio":0.24618948106796826,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":551},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:20:58.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:20:58.712 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:20:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:20:59.574 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:20:59.643 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-11T05:20:57.580275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280269460,"num_objects":68429,"num_object_clones":0,"num_object_copies":136858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213270,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8820,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1756416,"kb_used_data":722888,"kb_used_omap":12,"kb_used_meta":1033459,"kb_avail":281359104,"statfs":{"total":289910292480,"available":288111722496,"internally_reserved":0,"allocated":740237312,"data_stored":739365974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1058262017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,5,15,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27127808,"num_objects":-6623,"num_object_clones":0,"num_object_copies":-13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6623,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003908"},"pg_stats":[{"pgid":"2.f","version":"47'15866","reported_seq":15949,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.220758+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:57.220758+0000","last_peered":"2024-09-11T05:20:57.220758+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:57.220758+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17711104,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15866,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":649}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15677","reported_seq":15785,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.221415+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:57.221415+0000","last_peered":"2024-09-11T05:20:57.221415+0000","last_clean":"2024-09-11T05:20:57.221415+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:57.221415+0000","last_undegraded":"2024-09-11T05:20:57.221415+0000","last_fullsized":"2024-09-11T05:20:57.221415+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17297408,"num_objects":4223,"num_object_clones":0,"num_object_copies":8446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14075,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15486","reported_seq":19091,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.216612+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:57.216612+0000","last_peered":"2024-09-11T05:20:57.216612+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:57.216612+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17330200,"num_objects":4232,"num_object_clones":0,"num_object_copies":8464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1759,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13889,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1736,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15751","reported_seq":15828,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.248389+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:57.248389+0000","last_peered":"2024-09-11T05:20:57.248389+0000","last_clean":"2024-09-11T05:20:57.248389+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:57.248389+0000","last_undegraded":"2024-09-11T05:20:57.248389+0000","last_fullsized":"2024-09-11T05:20:57.248389+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240064,"num_objects":4209,"num_object_clones":0,"num_object_copies":8418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15751,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15865","reported_seq":16000,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.256255+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:57.256255+0000","last_peered":"2024-09-11T05:20:57.256255+0000","last_clean":"2024-09-11T05:20:57.256255+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:57.256255+0000","last_undegraded":"2024-09-11T05:20:57.256255+0000","last_fullsized":"2024-09-11T05:20:57.256255+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12699,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15643","reported_seq":22994,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:54.754795+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:54.754795+0000","last_peered":"2024-09-11T05:20:54.754795+0000","last_clean":"2024-09-11T05:20:54.754795+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:54.754795+0000","last_undegraded":"2024-09-11T05:20:54.754795+0000","last_fullsized":"2024-09-11T05:20:54.754795+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17989632,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12187,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15853","reported_seq":15991,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.227604+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:57.227604+0000","last_peered":"2024-09-11T05:20:57.227604+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:57.227604+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17813504,"num_objects":4349,"num_object_clones":0,"num_object_copies":8698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13018,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":773}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15583","reported_seq":15713,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.257959+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:57.257959+0000","last_peered":"2024-09-11T05:20:57.257959+0000","last_clean":"2024-09-11T05:20:57.257959+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:57.257959+0000","last_undegraded":"2024-09-11T05:20:57.257959+0000","last_fullsized":"2024-09-11T05:20:57.257959+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17616896,"num_objects":4301,"num_object_clones":0,"num_object_copies":8602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12886,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15678","reported_seq":15815,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.187733+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:57.187733+0000","last_peered":"2024-09-11T05:20:57.187733+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:57.187733+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17584128,"num_objects":4293,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13746,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":768}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15421","reported_seq":15592,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.194476+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:57.194476+0000","last_peered":"2024-09-11T05:20:57.194476+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:57.194476+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412096,"num_objects":4251,"num_object_clones":0,"num_object_copies":8502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12174,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15729","reported_seq":15852,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.189074+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:57.189074+0000","last_peered":"2024-09-11T05:20:57.189074+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:57.189074+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13758,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15462","reported_seq":22553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.253003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:57.253003+0000","last_peered":"2024-09-11T05:20:57.253003+0000","last_clean":"2024-09-11T05:20:57.253003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:57.253003+0000","last_undegraded":"2024-09-11T05:20:57.253003+0000","last_fullsized":"2024-09-11T05:20:57.253003+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17391616,"num_objects":4246,"num_object_clones":0,"num_object_copies":8492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13365,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15882","reported_seq":16019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.237156+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:57.237156+0000","last_peered":"2024-09-11T05:20:57.237156+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:57.237156+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17457152,"num_objects":4262,"num_object_clones":0,"num_object_copies":8524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12915,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":639}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15413","reported_seq":15550,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.320261+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:57.320261+0000","last_peered":"2024-09-11T05:20:57.320261+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:57.320261+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371136,"num_objects":4241,"num_object_clones":0,"num_object_copies":8482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12279,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15514","reported_seq":15655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.308719+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:57.308719+0000","last_peered":"2024-09-11T05:20:57.308719+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:57.308719+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17555456,"num_objects":4286,"num_object_clones":0,"num_object_copies":8572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12272,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15381","reported_seq":15530,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.287723+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:57.287723+0000","last_peered":"2024-09-11T05:20:57.287723+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:57.287723+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240681,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":695}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280269441,"num_objects":68427,"num_object_clones":0,"num_object_copies":136854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68427,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213268,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8819,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":737308672,"data_stored":737230948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549204,"kb_used_data":250944,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93822636,"statfs":{"total":96636764160,"available":96074379264,"internally_reserved":0,"allocated":256966656,"data_stored":256697285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552648,"kb_used_data":209976,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93819192,"statfs":{"total":96636764160,"available":96070852608,"internally_reserved":0,"allocated":215015424,"data_stored":214682833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215205,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654564,"kb_used_data":261968,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93717276,"statfs":{"total":96636764160,"available":95966490624,"internally_reserved":0,"allocated":268255232,"data_stored":267985856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267448320,"data_stored":267444841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213561344,"data_stored":213494650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":256299008,"data_stored":256291457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:02.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:03.473 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:03.473 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:03.538 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-11T05:21:01.581257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279667348,"num_objects":68282,"num_object_clones":0,"num_object_copies":136564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68282,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213417,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8820,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160954,"ondisk_log_size":160954,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1753820,"kb_used_data":716132,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281361700,"statfs":{"total":289910292480,"available":288114380800,"internally_reserved":0,"allocated":733319168,"data_stored":732447830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,5,14,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19156992,"num_objects":-4677,"num_object_clones":0,"num_object_copies":-9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4677,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004061"},"pg_stats":[{"pgid":"2.f","version":"47'15866","reported_seq":15949,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.220758+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:20:57.220758+0000","last_peered":"2024-09-11T05:20:57.220758+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:20:57.220758+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17711104,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15866,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":649}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15677","reported_seq":15785,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.221415+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:20:57.221415+0000","last_peered":"2024-09-11T05:20:57.221415+0000","last_clean":"2024-09-11T05:20:57.221415+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:20:57.221415+0000","last_undegraded":"2024-09-11T05:20:57.221415+0000","last_fullsized":"2024-09-11T05:20:57.221415+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17297408,"num_objects":4223,"num_object_clones":0,"num_object_copies":8446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14075,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15486","reported_seq":19091,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.216612+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:20:57.216612+0000","last_peered":"2024-09-11T05:20:57.216612+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:20:57.216612+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17330200,"num_objects":4232,"num_object_clones":0,"num_object_copies":8464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1759,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13889,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1736,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15751","reported_seq":15828,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.248389+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:20:57.248389+0000","last_peered":"2024-09-11T05:20:57.248389+0000","last_clean":"2024-09-11T05:20:57.248389+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:20:57.248389+0000","last_undegraded":"2024-09-11T05:20:57.248389+0000","last_fullsized":"2024-09-11T05:20:57.248389+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240064,"num_objects":4209,"num_object_clones":0,"num_object_copies":8418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15751,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15865","reported_seq":16000,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.256255+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:20:57.256255+0000","last_peered":"2024-09-11T05:20:57.256255+0000","last_clean":"2024-09-11T05:20:57.256255+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:20:57.256255+0000","last_undegraded":"2024-09-11T05:20:57.256255+0000","last_fullsized":"2024-09-11T05:20:57.256255+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12699,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15790","reported_seq":23141,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:59.981333+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:59.981333+0000","last_peered":"2024-09-11T05:20:59.981333+0000","last_clean":"2024-09-11T05:20:59.981333+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:59.981333+0000","last_undegraded":"2024-09-11T05:20:59.981333+0000","last_fullsized":"2024-09-11T05:20:59.981333+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17387520,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15853","reported_seq":15991,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.227604+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:20:57.227604+0000","last_peered":"2024-09-11T05:20:57.227604+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:20:57.227604+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17813504,"num_objects":4349,"num_object_clones":0,"num_object_copies":8698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13018,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":773}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15583","reported_seq":15713,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.257959+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:20:57.257959+0000","last_peered":"2024-09-11T05:20:57.257959+0000","last_clean":"2024-09-11T05:20:57.257959+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:20:57.257959+0000","last_undegraded":"2024-09-11T05:20:57.257959+0000","last_fullsized":"2024-09-11T05:20:57.257959+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17616896,"num_objects":4301,"num_object_clones":0,"num_object_copies":8602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12886,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15678","reported_seq":15815,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.187733+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:20:57.187733+0000","last_peered":"2024-09-11T05:20:57.187733+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:20:57.187733+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17584128,"num_objects":4293,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13746,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":768}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15421","reported_seq":15592,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.194476+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:20:57.194476+0000","last_peered":"2024-09-11T05:20:57.194476+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:20:57.194476+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412096,"num_objects":4251,"num_object_clones":0,"num_object_copies":8502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12174,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15729","reported_seq":15852,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.189074+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:20:57.189074+0000","last_peered":"2024-09-11T05:20:57.189074+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:20:57.189074+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17629184,"num_objects":4304,"num_object_clones":0,"num_object_copies":8608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13758,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15462","reported_seq":22553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:57.253003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:20:57.253003+0000","last_peered":"2024-09-11T05:20:57.253003+0000","last_clean":"2024-09-11T05:20:57.253003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:20:57.253003+0000","last_undegraded":"2024-09-11T05:20:57.253003+0000","last_fullsized":"2024-09-11T05:20:57.253003+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17391616,"num_objects":4246,"num_object_clones":0,"num_object_copies":8492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13365,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15882","reported_seq":16019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.237156+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:20:57.237156+0000","last_peered":"2024-09-11T05:20:57.237156+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:20:57.237156+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17457152,"num_objects":4262,"num_object_clones":0,"num_object_copies":8524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12915,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":639}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15413","reported_seq":15550,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.320261+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:20:57.320261+0000","last_peered":"2024-09-11T05:20:57.320261+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:20:57.320261+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371136,"num_objects":4241,"num_object_clones":0,"num_object_copies":8482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12279,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15514","reported_seq":15655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.308719+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:20:57.308719+0000","last_peered":"2024-09-11T05:20:57.308719+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:20:57.308719+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17555456,"num_objects":4286,"num_object_clones":0,"num_object_copies":8572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12272,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15381","reported_seq":15530,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:20:57.287723+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:20:57.287723+0000","last_peered":"2024-09-11T05:20:57.287723+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:20:57.287723+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240681,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":695}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279667329,"num_objects":68280,"num_object_clones":0,"num_object_copies":136560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213415,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8819,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":730370048,"data_stored":730292324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549204,"kb_used_data":250944,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93822636,"statfs":{"total":96636764160,"available":96074379264,"internally_reserved":0,"allocated":256966656,"data_stored":256697285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550052,"kb_used_data":203220,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93821788,"statfs":{"total":96636764160,"available":96073510912,"internally_reserved":0,"allocated":208097280,"data_stored":207764689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215205,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654564,"kb_used_data":261968,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93717276,"statfs":{"total":96636764160,"available":95966490624,"internally_reserved":0,"allocated":268255232,"data_stored":267985856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267448320,"data_stored":267444841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206622720,"data_stored":206556026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":256299008,"data_stored":256291457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:03.539 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:04.416 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:04.417 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:04.489 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-11T05:21:03.581707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270938772,"num_objects":66151,"num_object_clones":0,"num_object_copies":132302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215548,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8909,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160885,"ondisk_log_size":160885,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1750908,"kb_used_data":704900,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281364612,"statfs":{"total":289910292480,"available":288117362688,"internally_reserved":0,"allocated":721817600,"data_stored":720946262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1071041537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,9,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27316224,"num_objects":-6669,"num_object_clones":0,"num_object_copies":-13338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.f","version":"47'16009","reported_seq":16092,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.407179+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:02.407179+0000","last_peered":"2024-09-11T05:21:02.407179+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:02.407179+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125376,"num_objects":4181,"num_object_clones":0,"num_object_copies":8362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16009,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":506}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15794","reported_seq":15902,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.238529+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:02.238529+0000","last_peered":"2024-09-11T05:21:02.238529+0000","last_clean":"2024-09-11T05:21:02.238529+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:02.238529+0000","last_undegraded":"2024-09-11T05:21:02.238529+0000","last_fullsized":"2024-09-11T05:21:02.238529+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16818176,"num_objects":4106,"num_object_clones":0,"num_object_copies":8212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14192,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15630","reported_seq":19413,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.259246+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:02.259246+0000","last_peered":"2024-09-11T05:21:02.259246+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:02.259246+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740376,"num_objects":4088,"num_object_clones":0,"num_object_copies":8176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4088,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14033,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1825,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2418}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15882","reported_seq":15959,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.263881+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:02.263881+0000","last_peered":"2024-09-11T05:21:02.263881+0000","last_clean":"2024-09-11T05:21:02.263881+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:02.263881+0000","last_undegraded":"2024-09-11T05:21:02.263881+0000","last_fullsized":"2024-09-11T05:21:02.263881+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16703488,"num_objects":4078,"num_object_clones":0,"num_object_copies":8156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15882,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16017","reported_seq":16152,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.204339+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:02.204339+0000","last_peered":"2024-09-11T05:21:02.204339+0000","last_clean":"2024-09-11T05:21:02.204339+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:02.204339+0000","last_undegraded":"2024-09-11T05:21:02.204339+0000","last_fullsized":"2024-09-11T05:21:02.204339+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17006592,"num_objects":4152,"num_object_clones":0,"num_object_copies":8304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12851,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15790","reported_seq":23141,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:59.981333+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:20:59.981333+0000","last_peered":"2024-09-11T05:20:59.981333+0000","last_clean":"2024-09-11T05:20:59.981333+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:20:59.981333+0000","last_undegraded":"2024-09-11T05:20:59.981333+0000","last_fullsized":"2024-09-11T05:20:59.981333+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17387520,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15992","reported_seq":16130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.240680+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:02.240680+0000","last_peered":"2024-09-11T05:21:02.240680+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:02.240680+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244160,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13157,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15714","reported_seq":15844,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.300675+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:02.300675+0000","last_peered":"2024-09-11T05:21:02.300675+0000","last_clean":"2024-09-11T05:21:02.300675+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:02.300675+0000","last_undegraded":"2024-09-11T05:21:02.300675+0000","last_fullsized":"2024-09-11T05:21:02.300675+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17080320,"num_objects":4170,"num_object_clones":0,"num_object_copies":8340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13017,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15824","reported_seq":15961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.255496+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:02.255496+0000","last_peered":"2024-09-11T05:21:02.255496+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:02.255496+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16986112,"num_objects":4147,"num_object_clones":0,"num_object_copies":8294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13892,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":622}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15562","reported_seq":15733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.373376+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:02.373376+0000","last_peered":"2024-09-11T05:21:02.373376+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:02.373376+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12315,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15885","reported_seq":16007,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.281238+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:02.281238+0000","last_peered":"2024-09-11T05:21:02.281238+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:02.281238+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16990208,"num_objects":4148,"num_object_clones":0,"num_object_copies":8296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13914,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15590","reported_seq":22681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.231960+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:02.231960+0000","last_peered":"2024-09-11T05:21:02.231960+0000","last_clean":"2024-09-11T05:21:02.231960+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:02.231960+0000","last_undegraded":"2024-09-11T05:21:02.231960+0000","last_fullsized":"2024-09-11T05:21:02.231960+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16867328,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16027","reported_seq":16164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.348538+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:02.348538+0000","last_peered":"2024-09-11T05:21:02.348538+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:02.348538+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863232,"num_objects":4117,"num_object_clones":0,"num_object_copies":8234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13060,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15580","reported_seq":15719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.399668+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:02.399668+0000","last_peered":"2024-09-11T05:21:02.399668+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:02.399668+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12446,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15661","reported_seq":15802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.279759+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:02.279759+0000","last_peered":"2024-09-11T05:21:02.279759+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:02.279759+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953344,"num_objects":4139,"num_object_clones":0,"num_object_copies":8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12419,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15525","reported_seq":15674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.388716+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:02.388716+0000","last_peered":"2024-09-11T05:21:02.388716+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:02.388716+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650857,"num_objects":4066,"num_object_clones":0,"num_object_copies":8132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12532,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270938753,"num_objects":66149,"num_object_clones":0,"num_object_copies":132298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215546,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8908,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":718827520,"data_stored":718749796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160882,"ondisk_log_size":160882,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549056,"kb_used_data":246632,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93822784,"statfs":{"total":96636764160,"available":96074530816,"internally_reserved":0,"allocated":252551168,"data_stored":252281797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550052,"kb_used_data":203220,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93821788,"statfs":{"total":96636764160,"available":96073510912,"internally_reserved":0,"allocated":208097280,"data_stored":207764689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215206,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651800,"kb_used_data":255048,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93720040,"statfs":{"total":96636764160,"available":95969320960,"internally_reserved":0,"allocated":261169152,"data_stored":260899776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260427776,"data_stored":260424297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206622720,"data_stored":206556026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":251777024,"data_stored":251769473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:04.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:04.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16009", 'reported_seq': 16092, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.407179+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:02.407179+0000', 'last_peered': '2024-09-11T05:21:02.407179+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:02.407179+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17125376, 'num_objects': 4181, 'num_object_clones': 0, 'num_object_copies': 8362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16009, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 506}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:04.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:04.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15630", 'reported_seq': 19413, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.259246+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:02.259246+0000', 'last_peered': '2024-09-11T05:21:02.259246+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:02.259246+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16740376, 'num_objects': 4088, 'num_object_clones': 0, 'num_object_copies': 8176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1670, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4088, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14033, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1825, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2418}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:04.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:04.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15992", 'reported_seq': 16130, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.240680+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:02.240680+0000', 'last_peered': '2024-09-11T05:21:02.240680+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:02.240680+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17244160, 'num_objects': 4210, 'num_object_clones': 0, 'num_object_copies': 8420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13157, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 634}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:04.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15824", 'reported_seq': 15961, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.255496+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:02.255496+0000', 'last_peered': '2024-09-11T05:21:02.255496+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:02.255496+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16986112, 'num_objects': 4147, 'num_object_clones': 0, 'num_object_copies': 8294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13892, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 622}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15562", 'reported_seq': 15733, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.373376+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:02.373376+0000', 'last_peered': '2024-09-11T05:21:02.373376+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:02.373376+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16834560, 'num_objects': 4110, 'num_object_clones': 0, 'num_object_copies': 8220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12315, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 668}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15885", 'reported_seq': 16007, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.281238+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:02.281238+0000', 'last_peered': '2024-09-11T05:21:02.281238+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:02.281238+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16990208, 'num_objects': 4148, 'num_object_clones': 0, 'num_object_copies': 8296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13914, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 626}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16027", 'reported_seq': 16164, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.348538+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:02.348538+0000', 'last_peered': '2024-09-11T05:21:02.348538+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:02.348538+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16863232, 'num_objects': 4117, 'num_object_clones': 0, 'num_object_copies': 8234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13060, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 494}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:04.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15580", 'reported_seq': 15719, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.399668+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:02.399668+0000', 'last_peered': '2024-09-11T05:21:02.399668+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:02.399668+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16687104, 'num_objects': 4074, 'num_object_clones': 0, 'num_object_copies': 8148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12446, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 579}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:04.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:04.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15661", 'reported_seq': 15802, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.279759+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:02.279759+0000', 'last_peered': '2024-09-11T05:21:02.279759+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:02.279759+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16953344, 'num_objects': 4139, 'num_object_clones': 0, 'num_object_copies': 8278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12419, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 595}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:04.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:04.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15525", 'reported_seq': 15674, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:02.388716+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:02.388716+0000', 'last_peered': '2024-09-11T05:21:02.388716+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:02.388716+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16650857, 'num_objects': 4066, 'num_object_clones': 0, 'num_object_copies': 8132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4066, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12532, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 551}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:04.493 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:05.447 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:05.448 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33604/132302 objects degraded (25.399%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":543,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66151,"data_bytes":270938772,"bytes_used":1792929792,"bytes_avail":288117362688,"bytes_total":289910292480,"degraded_objects":33604,"degraded_total":132302,"degraded_ratio":0.25399464860697496,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:05.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:05.511 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:06.541 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:06.615 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-11T05:21:05.582213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270336660,"num_objects":66004,"num_object_clones":0,"num_object_copies":132008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66004,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215695,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8909,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1735796,"kb_used_data":698108,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281379724,"statfs":{"total":289910292480,"available":288132837376,"internally_reserved":0,"allocated":714862592,"data_stored":713991254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,7,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19230720,"num_objects":-4695,"num_object_clones":0,"num_object_copies":-9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4695,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002823"},"pg_stats":[{"pgid":"2.f","version":"47'16009","reported_seq":16092,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.407179+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:02.407179+0000","last_peered":"2024-09-11T05:21:02.407179+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:02.407179+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125376,"num_objects":4181,"num_object_clones":0,"num_object_copies":8362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16009,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":506}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15794","reported_seq":15902,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.238529+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:02.238529+0000","last_peered":"2024-09-11T05:21:02.238529+0000","last_clean":"2024-09-11T05:21:02.238529+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:02.238529+0000","last_undegraded":"2024-09-11T05:21:02.238529+0000","last_fullsized":"2024-09-11T05:21:02.238529+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16818176,"num_objects":4106,"num_object_clones":0,"num_object_copies":8212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14192,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15630","reported_seq":19413,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.259246+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:02.259246+0000","last_peered":"2024-09-11T05:21:02.259246+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:02.259246+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740376,"num_objects":4088,"num_object_clones":0,"num_object_copies":8176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4088,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14033,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1825,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2418}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15882","reported_seq":15959,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.263881+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:02.263881+0000","last_peered":"2024-09-11T05:21:02.263881+0000","last_clean":"2024-09-11T05:21:02.263881+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:02.263881+0000","last_undegraded":"2024-09-11T05:21:02.263881+0000","last_fullsized":"2024-09-11T05:21:02.263881+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16703488,"num_objects":4078,"num_object_clones":0,"num_object_copies":8156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15882,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16017","reported_seq":16152,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.204339+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:02.204339+0000","last_peered":"2024-09-11T05:21:02.204339+0000","last_clean":"2024-09-11T05:21:02.204339+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:02.204339+0000","last_undegraded":"2024-09-11T05:21:02.204339+0000","last_fullsized":"2024-09-11T05:21:02.204339+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17006592,"num_objects":4152,"num_object_clones":0,"num_object_copies":8304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12851,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15937","reported_seq":23287,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:04.965898+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:04.965898+0000","last_peered":"2024-09-11T05:21:04.965898+0000","last_clean":"2024-09-11T05:21:04.965898+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:04.965898+0000","last_undegraded":"2024-09-11T05:21:04.965898+0000","last_fullsized":"2024-09-11T05:21:04.965898+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12481,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'15992","reported_seq":16130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.240680+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:02.240680+0000","last_peered":"2024-09-11T05:21:02.240680+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:02.240680+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244160,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13157,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15714","reported_seq":15844,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.300675+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:02.300675+0000","last_peered":"2024-09-11T05:21:02.300675+0000","last_clean":"2024-09-11T05:21:02.300675+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:02.300675+0000","last_undegraded":"2024-09-11T05:21:02.300675+0000","last_fullsized":"2024-09-11T05:21:02.300675+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17080320,"num_objects":4170,"num_object_clones":0,"num_object_copies":8340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13017,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15824","reported_seq":15961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.255496+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:02.255496+0000","last_peered":"2024-09-11T05:21:02.255496+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:02.255496+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16986112,"num_objects":4147,"num_object_clones":0,"num_object_copies":8294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13892,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":622}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15562","reported_seq":15733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.373376+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:02.373376+0000","last_peered":"2024-09-11T05:21:02.373376+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:02.373376+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12315,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15885","reported_seq":16007,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.281238+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:02.281238+0000","last_peered":"2024-09-11T05:21:02.281238+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:02.281238+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16990208,"num_objects":4148,"num_object_clones":0,"num_object_copies":8296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13914,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15590","reported_seq":22681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:02.231960+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:02.231960+0000","last_peered":"2024-09-11T05:21:02.231960+0000","last_clean":"2024-09-11T05:21:02.231960+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:02.231960+0000","last_undegraded":"2024-09-11T05:21:02.231960+0000","last_fullsized":"2024-09-11T05:21:02.231960+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16867328,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16027","reported_seq":16164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.348538+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:02.348538+0000","last_peered":"2024-09-11T05:21:02.348538+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:02.348538+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863232,"num_objects":4117,"num_object_clones":0,"num_object_copies":8234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13060,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15580","reported_seq":15719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.399668+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:02.399668+0000","last_peered":"2024-09-11T05:21:02.399668+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:02.399668+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12446,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15661","reported_seq":15802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.279759+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:02.279759+0000","last_peered":"2024-09-11T05:21:02.279759+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:02.279759+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953344,"num_objects":4139,"num_object_clones":0,"num_object_copies":8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12419,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15525","reported_seq":15674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:02.388716+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:02.388716+0000","last_peered":"2024-09-11T05:21:02.388716+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:02.388716+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650857,"num_objects":4066,"num_object_clones":0,"num_object_copies":8132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12532,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270336641,"num_objects":66002,"num_object_clones":0,"num_object_copies":132004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215693,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8908,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711749632,"data_stored":711671908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549056,"kb_used_data":246632,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93822784,"statfs":{"total":96636764160,"available":96074530816,"internally_reserved":0,"allocated":252551168,"data_stored":252281797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534940,"kb_used_data":196428,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93836900,"statfs":{"total":96636764160,"available":96088985600,"internally_reserved":0,"allocated":201142272,"data_stored":200809681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215206,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651800,"kb_used_data":255048,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93720040,"statfs":{"total":96636764160,"available":95969320960,"internally_reserved":0,"allocated":261169152,"data_stored":260899776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260427776,"data_stored":260424297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199544832,"data_stored":199478138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":251777024,"data_stored":251769473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:09.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:10.389 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:10.391 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:10.426 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-11T05:21:09.583146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261677716,"num_objects":63890,"num_object_clones":0,"num_object_copies":127780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63890,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217809,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9011,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720420,"kb_used_data":686896,"kb_used_omap":12,"kb_used_meta":1033459,"kb_avail":281395100,"statfs":{"total":289910292480,"available":288148582400,"internally_reserved":0,"allocated":703381504,"data_stored":702510166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1058262017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,5,4,5,2,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18591744,"num_objects":-4539,"num_object_clones":0,"num_object_copies":-9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002870"},"pg_stats":[{"pgid":"2.f","version":"47'16147","reported_seq":16231,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.378857+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:07.378857+0000","last_peered":"2024-09-11T05:21:07.378857+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:07.378857+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15945","reported_seq":16053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.239177+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:07.239177+0000","last_peered":"2024-09-11T05:21:07.239177+0000","last_clean":"2024-09-11T05:21:07.239177+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:07.239177+0000","last_undegraded":"2024-09-11T05:21:07.239177+0000","last_fullsized":"2024-09-11T05:21:07.239177+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14343,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15774","reported_seq":19761,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.276257+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:07.276257+0000","last_peered":"2024-09-11T05:21:07.276257+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:07.276257+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150552,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14177,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1927,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16040","reported_seq":16117,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.297547+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:07.297547+0000","last_peered":"2024-09-11T05:21:07.297547+0000","last_clean":"2024-09-11T05:21:07.297547+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:07.297547+0000","last_undegraded":"2024-09-11T05:21:07.297547+0000","last_fullsized":"2024-09-11T05:21:07.297547+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16040,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16151","reported_seq":16286,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.331999+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:07.331999+0000","last_peered":"2024-09-11T05:21:07.331999+0000","last_clean":"2024-09-11T05:21:07.331999+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:07.331999+0000","last_undegraded":"2024-09-11T05:21:07.331999+0000","last_fullsized":"2024-09-11T05:21:07.331999+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457728,"num_objects":4018,"num_object_clones":0,"num_object_copies":8036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12985,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15937","reported_seq":23287,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:04.965898+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:04.965898+0000","last_peered":"2024-09-11T05:21:04.965898+0000","last_clean":"2024-09-11T05:21:04.965898+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:04.965898+0000","last_undegraded":"2024-09-11T05:21:04.965898+0000","last_fullsized":"2024-09-11T05:21:04.965898+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12481,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16154","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.304148+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:07.304148+0000","last_peered":"2024-09-11T05:21:07.304148+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:07.304148+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15858","reported_seq":15988,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.380077+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:07.380077+0000","last_peered":"2024-09-11T05:21:07.380077+0000","last_clean":"2024-09-11T05:21:07.380077+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:07.380077+0000","last_undegraded":"2024-09-11T05:21:07.380077+0000","last_fullsized":"2024-09-11T05:21:07.380077+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490496,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13161,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15954","reported_seq":16090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.268678+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:07.268678+0000","last_peered":"2024-09-11T05:21:07.268678+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:07.268678+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16453632,"num_objects":4017,"num_object_clones":0,"num_object_copies":8034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14022,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15700","reported_seq":15871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.353212+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:07.353212+0000","last_peered":"2024-09-11T05:21:07.353212+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:07.353212+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16269312,"num_objects":3972,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12453,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":530}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16018","reported_seq":16141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.302819+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:07.302819+0000","last_peered":"2024-09-11T05:21:07.302819+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:07.302819+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16445440,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14047,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15727","reported_seq":22818,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.221280+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:07.221280+0000","last_peered":"2024-09-11T05:21:07.221280+0000","last_clean":"2024-09-11T05:21:07.221280+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:07.221280+0000","last_undegraded":"2024-09-11T05:21:07.221280+0000","last_fullsized":"2024-09-11T05:21:07.221280+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13630,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16155","reported_seq":16292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.403194+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:07.403194+0000","last_peered":"2024-09-11T05:21:07.403194+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:07.403194+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16338944,"num_objects":3989,"num_object_clones":0,"num_object_copies":7978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13188,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15716","reported_seq":15855,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.347667+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:07.347667+0000","last_peered":"2024-09-11T05:21:07.347667+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:07.347667+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12582,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15810","reported_seq":15948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.366882+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:07.366882+0000","last_peered":"2024-09-11T05:21:07.366882+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:07.366882+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343040,"num_objects":3990,"num_object_clones":0,"num_object_copies":7980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12568,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15657","reported_seq":15806,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.331938+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:07.331938+0000","last_peered":"2024-09-11T05:21:07.331938+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:07.331938+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16110185,"num_objects":3934,"num_object_clones":0,"num_object_copies":7868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12664,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261677697,"num_objects":63888,"num_object_clones":0,"num_object_copies":127776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217807,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9010,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":700178432,"data_stored":700100708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548832,"kb_used_data":242256,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93823008,"statfs":{"total":96636764160,"available":96074760192,"internally_reserved":0,"allocated":248070144,"data_stored":247800773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534940,"kb_used_data":196428,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93836900,"statfs":{"total":96636764160,"available":96088985600,"internally_reserved":0,"allocated":201142272,"data_stored":200809681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215207,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636648,"kb_used_data":248212,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93735192,"statfs":{"total":96636764160,"available":95984836608,"internally_reserved":0,"allocated":254169088,"data_stored":253899712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,1,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":253468672,"data_stored":253465193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199544832,"data_stored":199478138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247164928,"data_stored":247157377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:10.428 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:11.259 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:11.260 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:11.315 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-11T05:21:09.583146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261677716,"num_objects":63890,"num_object_clones":0,"num_object_copies":127780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63890,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217809,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9011,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720420,"kb_used_data":686896,"kb_used_omap":12,"kb_used_meta":1033459,"kb_avail":281395100,"statfs":{"total":289910292480,"available":288148582400,"internally_reserved":0,"allocated":703381504,"data_stored":702510166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1058262017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,5,4,5,2,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18591744,"num_objects":-4539,"num_object_clones":0,"num_object_copies":-9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002870"},"pg_stats":[{"pgid":"2.f","version":"47'16147","reported_seq":16231,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.378857+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:07.378857+0000","last_peered":"2024-09-11T05:21:07.378857+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:07.378857+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15945","reported_seq":16053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.239177+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:07.239177+0000","last_peered":"2024-09-11T05:21:07.239177+0000","last_clean":"2024-09-11T05:21:07.239177+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:07.239177+0000","last_undegraded":"2024-09-11T05:21:07.239177+0000","last_fullsized":"2024-09-11T05:21:07.239177+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14343,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15774","reported_seq":19761,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.276257+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:07.276257+0000","last_peered":"2024-09-11T05:21:07.276257+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:07.276257+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150552,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14177,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1927,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16040","reported_seq":16117,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.297547+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:07.297547+0000","last_peered":"2024-09-11T05:21:07.297547+0000","last_clean":"2024-09-11T05:21:07.297547+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:07.297547+0000","last_undegraded":"2024-09-11T05:21:07.297547+0000","last_fullsized":"2024-09-11T05:21:07.297547+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16040,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16151","reported_seq":16286,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.331999+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:07.331999+0000","last_peered":"2024-09-11T05:21:07.331999+0000","last_clean":"2024-09-11T05:21:07.331999+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:07.331999+0000","last_undegraded":"2024-09-11T05:21:07.331999+0000","last_fullsized":"2024-09-11T05:21:07.331999+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457728,"num_objects":4018,"num_object_clones":0,"num_object_copies":8036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12985,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15937","reported_seq":23287,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:04.965898+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:04.965898+0000","last_peered":"2024-09-11T05:21:04.965898+0000","last_clean":"2024-09-11T05:21:04.965898+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:04.965898+0000","last_undegraded":"2024-09-11T05:21:04.965898+0000","last_fullsized":"2024-09-11T05:21:04.965898+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12481,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16154","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.304148+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:07.304148+0000","last_peered":"2024-09-11T05:21:07.304148+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:07.304148+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15858","reported_seq":15988,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.380077+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:07.380077+0000","last_peered":"2024-09-11T05:21:07.380077+0000","last_clean":"2024-09-11T05:21:07.380077+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:07.380077+0000","last_undegraded":"2024-09-11T05:21:07.380077+0000","last_fullsized":"2024-09-11T05:21:07.380077+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490496,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13161,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15954","reported_seq":16090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.268678+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:07.268678+0000","last_peered":"2024-09-11T05:21:07.268678+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:07.268678+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16453632,"num_objects":4017,"num_object_clones":0,"num_object_copies":8034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14022,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15700","reported_seq":15871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.353212+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:07.353212+0000","last_peered":"2024-09-11T05:21:07.353212+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:07.353212+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16269312,"num_objects":3972,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12453,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":530}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16018","reported_seq":16141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.302819+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:07.302819+0000","last_peered":"2024-09-11T05:21:07.302819+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:07.302819+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16445440,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14047,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15727","reported_seq":22818,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.221280+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:07.221280+0000","last_peered":"2024-09-11T05:21:07.221280+0000","last_clean":"2024-09-11T05:21:07.221280+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:07.221280+0000","last_undegraded":"2024-09-11T05:21:07.221280+0000","last_fullsized":"2024-09-11T05:21:07.221280+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13630,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16155","reported_seq":16292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.403194+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:07.403194+0000","last_peered":"2024-09-11T05:21:07.403194+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:07.403194+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16338944,"num_objects":3989,"num_object_clones":0,"num_object_copies":7978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13188,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15716","reported_seq":15855,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.347667+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:07.347667+0000","last_peered":"2024-09-11T05:21:07.347667+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:07.347667+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12582,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15810","reported_seq":15948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.366882+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:07.366882+0000","last_peered":"2024-09-11T05:21:07.366882+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:07.366882+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343040,"num_objects":3990,"num_object_clones":0,"num_object_copies":7980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12568,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15657","reported_seq":15806,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.331938+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:07.331938+0000","last_peered":"2024-09-11T05:21:07.331938+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:07.331938+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16110185,"num_objects":3934,"num_object_clones":0,"num_object_copies":7868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12664,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261677697,"num_objects":63888,"num_object_clones":0,"num_object_copies":127776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217807,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9010,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":700178432,"data_stored":700100708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548832,"kb_used_data":242256,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93823008,"statfs":{"total":96636764160,"available":96074760192,"internally_reserved":0,"allocated":248070144,"data_stored":247800773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534940,"kb_used_data":196428,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93836900,"statfs":{"total":96636764160,"available":96088985600,"internally_reserved":0,"allocated":201142272,"data_stored":200809681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346615467},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215207,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636648,"kb_used_data":248212,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93735192,"statfs":{"total":96636764160,"available":95984836608,"internally_reserved":0,"allocated":254169088,"data_stored":253899712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,1,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":253468672,"data_stored":253465193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199544832,"data_stored":199478138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247164928,"data_stored":247157377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:11.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:11.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16147", 'reported_seq': 16231, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.378857+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:07.378857+0000', 'last_peered': '2024-09-11T05:21:07.378857+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:07.378857+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16560128, 'num_objects': 4043, 'num_object_clones': 0, 'num_object_copies': 8086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4043, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16147, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 368}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:11.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:11.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15774", 'reported_seq': 19761, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.276257+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:07.276257+0000', 'last_peered': '2024-09-11T05:21:07.276257+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:07.276257+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16150552, 'num_objects': 3944, 'num_object_clones': 0, 'num_object_copies': 7888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3944, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14177, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1927, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'16154", 'reported_seq': 16286, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.304148+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:07.304148+0000', 'last_peered': '2024-09-11T05:21:07.304148+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:07.304148+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16580608, 'num_objects': 4048, 'num_object_clones': 0, 'num_object_copies': 8096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13319, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 472}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15954", 'reported_seq': 16090, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.268678+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:07.268678+0000', 'last_peered': '2024-09-11T05:21:07.268678+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:07.268678+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16453632, 'num_objects': 4017, 'num_object_clones': 0, 'num_object_copies': 8034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14022, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 492}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15700", 'reported_seq': 15871, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.353212+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:07.353212+0000', 'last_peered': '2024-09-11T05:21:07.353212+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:07.353212+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16269312, 'num_objects': 3972, 'num_object_clones': 0, 'num_object_copies': 7944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12453, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 530}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16018", 'reported_seq': 16141, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.302819+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:07.302819+0000', 'last_peered': '2024-09-11T05:21:07.302819+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:07.302819+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16445440, 'num_objects': 4015, 'num_object_clones': 0, 'num_object_copies': 8030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14047, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 493}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:11.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16155", 'reported_seq': 16292, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.403194+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:07.403194+0000', 'last_peered': '2024-09-11T05:21:07.403194+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:07.403194+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16338944, 'num_objects': 3989, 'num_object_clones': 0, 'num_object_copies': 7978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13188, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 366}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:11.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:11.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15716", 'reported_seq': 15855, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.347667+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:07.347667+0000', 'last_peered': '2024-09-11T05:21:07.347667+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:07.347667+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16130048, 'num_objects': 3938, 'num_object_clones': 0, 'num_object_copies': 7876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12582, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:11.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:11.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15810", 'reported_seq': 15948, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.366882+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:07.366882+0000', 'last_peered': '2024-09-11T05:21:07.366882+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:07.366882+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16343040, 'num_objects': 3990, 'num_object_clones': 0, 'num_object_copies': 7980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3990, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12568, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 446}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:11.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:11.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15657", 'reported_seq': 15806, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:07.331938+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:07.331938+0000', 'last_peered': '2024-09-11T05:21:07.331938+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:07.331938+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16110185, 'num_objects': 3934, 'num_object_clones': 0, 'num_object_copies': 7868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3934, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12664, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:11.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:12.522 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:12.522 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33502/127504 objects degraded (26.275%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":550,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":63752,"data_bytes":261112468,"bytes_used":1757810688,"bytes_avail":288152481792,"bytes_total":289910292480,"degraded_objects":33502,"degraded_total":127504,"degraded_ratio":0.26275254109674989,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:12.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:12.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:13.466 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:13.469 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:13.516 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-11T05:21:11.583641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261112468,"num_objects":63752,"num_object_clones":0,"num_object_copies":127504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217947,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9011,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1716612,"kb_used_data":678924,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281398908,"statfs":{"total":289910292480,"available":288152481792,"internally_reserved":0,"allocated":695218176,"data_stored":694346838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,6,2,5,5,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19156992,"num_objects":-4677,"num_object_clones":0,"num_object_copies":-9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4677,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002869"},"pg_stats":[{"pgid":"2.f","version":"47'16147","reported_seq":16231,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.378857+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:07.378857+0000","last_peered":"2024-09-11T05:21:07.378857+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:07.378857+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15945","reported_seq":16053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.239177+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:07.239177+0000","last_peered":"2024-09-11T05:21:07.239177+0000","last_clean":"2024-09-11T05:21:07.239177+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:07.239177+0000","last_undegraded":"2024-09-11T05:21:07.239177+0000","last_fullsized":"2024-09-11T05:21:07.239177+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14343,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15774","reported_seq":19761,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.276257+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:07.276257+0000","last_peered":"2024-09-11T05:21:07.276257+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:07.276257+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150552,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14177,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1927,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2376}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16040","reported_seq":16117,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.297547+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:07.297547+0000","last_peered":"2024-09-11T05:21:07.297547+0000","last_clean":"2024-09-11T05:21:07.297547+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:07.297547+0000","last_undegraded":"2024-09-11T05:21:07.297547+0000","last_fullsized":"2024-09-11T05:21:07.297547+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16040,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16151","reported_seq":16286,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.331999+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:07.331999+0000","last_peered":"2024-09-11T05:21:07.331999+0000","last_clean":"2024-09-11T05:21:07.331999+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:07.331999+0000","last_undegraded":"2024-09-11T05:21:07.331999+0000","last_fullsized":"2024-09-11T05:21:07.331999+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457728,"num_objects":4018,"num_object_clones":0,"num_object_copies":8036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12985,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16075","reported_seq":23426,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:10.014546+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:10.014546+0000","last_peered":"2024-09-11T05:21:10.014546+0000","last_clean":"2024-09-11T05:21:10.014546+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:10.014546+0000","last_undegraded":"2024-09-11T05:21:10.014546+0000","last_fullsized":"2024-09-11T05:21:10.014546+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12619,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16154","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.304148+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:07.304148+0000","last_peered":"2024-09-11T05:21:07.304148+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:07.304148+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15858","reported_seq":15988,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.380077+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:07.380077+0000","last_peered":"2024-09-11T05:21:07.380077+0000","last_clean":"2024-09-11T05:21:07.380077+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:07.380077+0000","last_undegraded":"2024-09-11T05:21:07.380077+0000","last_fullsized":"2024-09-11T05:21:07.380077+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490496,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13161,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15954","reported_seq":16090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.268678+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:07.268678+0000","last_peered":"2024-09-11T05:21:07.268678+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:07.268678+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16453632,"num_objects":4017,"num_object_clones":0,"num_object_copies":8034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14022,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15700","reported_seq":15871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.353212+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:07.353212+0000","last_peered":"2024-09-11T05:21:07.353212+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:07.353212+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16269312,"num_objects":3972,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12453,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":530}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16018","reported_seq":16141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.302819+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:07.302819+0000","last_peered":"2024-09-11T05:21:07.302819+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:07.302819+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16445440,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14047,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15727","reported_seq":22818,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:07.221280+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:07.221280+0000","last_peered":"2024-09-11T05:21:07.221280+0000","last_clean":"2024-09-11T05:21:07.221280+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:07.221280+0000","last_undegraded":"2024-09-11T05:21:07.221280+0000","last_fullsized":"2024-09-11T05:21:07.221280+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13630,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16155","reported_seq":16292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.403194+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:07.403194+0000","last_peered":"2024-09-11T05:21:07.403194+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:07.403194+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16338944,"num_objects":3989,"num_object_clones":0,"num_object_copies":7978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13188,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15716","reported_seq":15855,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.347667+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:07.347667+0000","last_peered":"2024-09-11T05:21:07.347667+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:07.347667+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16130048,"num_objects":3938,"num_object_clones":0,"num_object_copies":7876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12582,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15810","reported_seq":15948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.366882+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:07.366882+0000","last_peered":"2024-09-11T05:21:07.366882+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:07.366882+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16343040,"num_objects":3990,"num_object_clones":0,"num_object_copies":7980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12568,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15657","reported_seq":15806,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:07.331938+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:07.331938+0000","last_peered":"2024-09-11T05:21:07.331938+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:07.331938+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16110185,"num_objects":3934,"num_object_clones":0,"num_object_copies":7868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12664,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261112449,"num_objects":63750,"num_object_clones":0,"num_object_copies":127500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217945,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9010,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":693182464,"data_stored":693104740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548832,"kb_used_data":242256,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93823008,"statfs":{"total":96636764160,"available":96074760192,"internally_reserved":0,"allocated":248070144,"data_stored":247800773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531132,"kb_used_data":188456,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93840708,"statfs":{"total":96636764160,"available":96092884992,"internally_reserved":0,"allocated":192978944,"data_stored":192646353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,3,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215207,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636648,"kb_used_data":248212,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93735192,"statfs":{"total":96636764160,"available":95984836608,"internally_reserved":0,"allocated":254169088,"data_stored":253899712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,1,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":253468672,"data_stored":253465193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192548864,"data_stored":192482170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247164928,"data_stored":247157377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:16.518 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:17.372 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:17.373 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:17.434 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-11T05:21:15.584558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":251810452,"num_objects":61481,"num_object_clones":0,"num_object_copies":122962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220218,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9105,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160955,"ondisk_log_size":160955,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1734576,"kb_used_data":660792,"kb_used_omap":12,"kb_used_meta":1073715,"kb_avail":281380944,"statfs":{"total":289910292480,"available":288134086656,"internally_reserved":0,"allocated":676651008,"data_stored":675779670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1099484161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,8,16,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19128320,"num_objects":-4670,"num_object_clones":0,"num_object_copies":-9340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4670,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002850"},"pg_stats":[{"pgid":"2.f","version":"47'16295","reported_seq":16379,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.357252+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:12.357252+0000","last_peered":"2024-09-11T05:21:12.357252+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:12.357252+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15953920,"num_objects":3895,"num_object_clones":0,"num_object_copies":7790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16295,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":220}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16084","reported_seq":16192,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:12.339792+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:12.339792+0000","last_peered":"2024-09-11T05:21:12.339792+0000","last_clean":"2024-09-11T05:21:12.339792+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:12.339792+0000","last_undegraded":"2024-09-11T05:21:12.339792+0000","last_fullsized":"2024-09-11T05:21:12.339792+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630336,"num_objects":3816,"num_object_clones":0,"num_object_copies":7632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14482,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15915","reported_seq":20090,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.334662+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:12.334662+0000","last_peered":"2024-09-11T05:21:12.334662+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:12.334662+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15573016,"num_objects":3803,"num_object_clones":0,"num_object_copies":7606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3803,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14318,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2021,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2329}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16191","reported_seq":16268,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:12.153901+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:12.153901+0000","last_peered":"2024-09-11T05:21:12.153901+0000","last_clean":"2024-09-11T05:21:12.153901+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:12.153901+0000","last_undegraded":"2024-09-11T05:21:12.153901+0000","last_fullsized":"2024-09-11T05:21:12.153901+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15437824,"num_objects":3769,"num_object_clones":0,"num_object_copies":7538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16191,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16277","reported_seq":16412,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:12.378649+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:12.378649+0000","last_peered":"2024-09-11T05:21:12.378649+0000","last_clean":"2024-09-11T05:21:12.378649+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:12.378649+0000","last_undegraded":"2024-09-11T05:21:12.378649+0000","last_fullsized":"2024-09-11T05:21:12.378649+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15941632,"num_objects":3892,"num_object_clones":0,"num_object_copies":7784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13111,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16226","reported_seq":23577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:14.996369+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:14.996369+0000","last_peered":"2024-09-11T05:21:14.996369+0000","last_clean":"2024-09-11T05:21:14.996369+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:14.996369+0000","last_undegraded":"2024-09-11T05:21:14.996369+0000","last_fullsized":"2024-09-11T05:21:14.996369+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15601664,"num_objects":3809,"num_object_clones":0,"num_object_copies":7618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12770,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16298","reported_seq":16436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.306656+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:12.306656+0000","last_peered":"2024-09-11T05:21:12.306656+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:12.306656+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15990784,"num_objects":3904,"num_object_clones":0,"num_object_copies":7808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13463,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16011","reported_seq":16141,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:12.404416+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:12.404416+0000","last_peered":"2024-09-11T05:21:12.404416+0000","last_clean":"2024-09-11T05:21:12.404416+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:12.404416+0000","last_undegraded":"2024-09-11T05:21:12.404416+0000","last_fullsized":"2024-09-11T05:21:12.404416+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15863808,"num_objects":3873,"num_object_clones":0,"num_object_copies":7746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13314,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16093","reported_seq":16230,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.284119+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:12.284119+0000","last_peered":"2024-09-11T05:21:12.284119+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:12.284119+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15884288,"num_objects":3878,"num_object_clones":0,"num_object_copies":7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14161,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15844","reported_seq":16011,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.301893+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:12.301893+0000","last_peered":"2024-09-11T05:21:12.301893+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:12.301893+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15679488,"num_objects":3828,"num_object_clones":0,"num_object_copies":7656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":386}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710714+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710714+0000","last_peered":"2024-09-11T05:19:25.710714+0000","last_clean":"2024-09-11T05:19:25.710714+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710714+0000","last_undegraded":"2024-09-11T05:19:25.710714+0000","last_fullsized":"2024-09-11T05:19:25.710714+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16159","reported_seq":16282,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.340532+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:12.340532+0000","last_peered":"2024-09-11T05:21:12.340532+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:12.340532+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15867904,"num_objects":3874,"num_object_clones":0,"num_object_copies":7748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14188,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":352}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15847","reported_seq":22938,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:12.275115+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:12.275115+0000","last_peered":"2024-09-11T05:21:12.275115+0000","last_clean":"2024-09-11T05:21:12.275115+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:12.275115+0000","last_undegraded":"2024-09-11T05:21:12.275115+0000","last_fullsized":"2024-09-11T05:21:12.275115+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13750,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16284","reported_seq":16421,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.359617+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:12.359617+0000","last_peered":"2024-09-11T05:21:12.359617+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:12.359617+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15810560,"num_objects":3860,"num_object_clones":0,"num_object_copies":7720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13317,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15870","reported_seq":16009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.431444+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:12.431444+0000","last_peered":"2024-09-11T05:21:12.431444+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:12.431444+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15499264,"num_objects":3784,"num_object_clones":0,"num_object_copies":7568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15952","reported_seq":16093,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.390225+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:12.390225+0000","last_peered":"2024-09-11T05:21:12.390225+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:12.390225+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15761408,"num_objects":3848,"num_object_clones":0,"num_object_copies":7696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12710,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15806","reported_seq":15955,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:12.394742+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:12.394742+0000","last_peered":"2024-09-11T05:21:12.394742+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:12.394742+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15499881,"num_objects":3785,"num_object_clones":0,"num_object_copies":7570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":270}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":251810433,"num_objects":61479,"num_object_clones":0,"num_object_copies":122958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220216,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9104,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":674607104,"data_stored":674529380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548544,"kb_used_data":237808,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93823296,"statfs":{"total":96636764160,"available":96075055104,"internally_reserved":0,"allocated":243515392,"data_stored":243246021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534164,"kb_used_data":181756,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93837676,"statfs":{"total":96636764160,"available":96089780224,"internally_reserved":0,"allocated":186118144,"data_stored":185785553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651868,"kb_used_data":241228,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93719972,"statfs":{"total":96636764160,"available":95969251328,"internally_reserved":0,"allocated":247017472,"data_stored":246748096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":246431744,"data_stored":246428265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185544704,"data_stored":185478010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":242630656,"data_stored":242623105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:17.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:18.260 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:18.263 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:18.303 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-11T05:21:17.585010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243016340,"num_objects":59334,"num_object_clones":0,"num_object_copies":118668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222365,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9212,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1718956,"kb_used_data":649324,"kb_used_omap":12,"kb_used_meta":1069555,"kb_avail":281396564,"statfs":{"total":289910292480,"available":288150081536,"internally_reserved":0,"allocated":664907776,"data_stored":664036438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1095224321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,6,11,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27320320,"num_objects":-6670,"num_object_clones":0,"num_object_copies":-13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6670,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"2.f","version":"47'16445","reported_seq":16527,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.378740+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:17.378740+0000","last_peered":"2024-09-11T05:21:17.378740+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:17.378740+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15339520,"num_objects":3745,"num_object_clones":0,"num_object_copies":7490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16445,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16219","reported_seq":16327,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.305089+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:17.305089+0000","last_peered":"2024-09-11T05:21:17.305089+0000","last_clean":"2024-09-11T05:21:17.305089+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:17.305089+0000","last_undegraded":"2024-09-11T05:21:17.305089+0000","last_fullsized":"2024-09-11T05:21:17.305089+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15077376,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14617,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16050","reported_seq":20439,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.271072+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:17.271072+0000","last_peered":"2024-09-11T05:21:17.271072+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:17.271072+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15020056,"num_objects":3668,"num_object_clones":0,"num_object_copies":7336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3668,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14453,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2128,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2301}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16334","reported_seq":16411,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.276468+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:17.276468+0000","last_peered":"2024-09-11T05:21:17.276468+0000","last_clean":"2024-09-11T05:21:17.276468+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:17.276468+0000","last_undegraded":"2024-09-11T05:21:17.276468+0000","last_fullsized":"2024-09-11T05:21:17.276468+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14852096,"num_objects":3626,"num_object_clones":0,"num_object_copies":7252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16424","reported_seq":16559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.466444+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:17.466444+0000","last_peered":"2024-09-11T05:21:17.466444+0000","last_clean":"2024-09-11T05:21:17.466444+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:17.466444+0000","last_undegraded":"2024-09-11T05:21:17.466444+0000","last_fullsized":"2024-09-11T05:21:17.466444+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15339520,"num_objects":3745,"num_object_clones":0,"num_object_copies":7490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13258,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16226","reported_seq":23577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:14.996369+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:14.996369+0000","last_peered":"2024-09-11T05:21:14.996369+0000","last_clean":"2024-09-11T05:21:14.996369+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:14.996369+0000","last_undegraded":"2024-09-11T05:21:14.996369+0000","last_fullsized":"2024-09-11T05:21:14.996369+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15601664,"num_objects":3809,"num_object_clones":0,"num_object_copies":7618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12770,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16460","reported_seq":16597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.446460+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:17.446460+0000","last_peered":"2024-09-11T05:21:17.446460+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:17.446460+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13625,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16150","reported_seq":16279,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.440929+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:17.440929+0000","last_peered":"2024-09-11T05:21:17.440929+0000","last_clean":"2024-09-11T05:21:17.440929+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:17.440929+0000","last_undegraded":"2024-09-11T05:21:17.440929+0000","last_fullsized":"2024-09-11T05:21:17.440929+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294464,"num_objects":3734,"num_object_clones":0,"num_object_copies":7468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13453,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16253","reported_seq":16389,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.267456+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:17.267456+0000","last_peered":"2024-09-11T05:21:17.267456+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:17.267456+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15228928,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14321,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15961","reported_seq":16132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.397200+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:17.397200+0000","last_peered":"2024-09-11T05:21:17.397200+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:17.397200+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15200256,"num_objects":3711,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12714,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16322","reported_seq":16445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.240098+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:17.240098+0000","last_peered":"2024-09-11T05:21:17.240098+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:17.240098+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15200256,"num_objects":3711,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14351,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":189}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15974","reported_seq":23065,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.292175+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:17.292175+0000","last_peered":"2024-09-11T05:21:17.292175+0000","last_clean":"2024-09-11T05:21:17.292175+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:17.292175+0000","last_undegraded":"2024-09-11T05:21:17.292175+0000","last_fullsized":"2024-09-11T05:21:17.292175+0000","mapping_epoch":44,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294464,"num_objects":3734,"num_object_clones":0,"num_object_copies":7468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13877,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16442","reported_seq":16579,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.412339+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:17.412339+0000","last_peered":"2024-09-11T05:21:17.412339+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:17.412339+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15163392,"num_objects":3702,"num_object_clones":0,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13475,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":79}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16011","reported_seq":16150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.453438+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:17.453438+0000","last_peered":"2024-09-11T05:21:17.453438+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:17.453438+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14921728,"num_objects":3643,"num_object_clones":0,"num_object_copies":7286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16082","reported_seq":16222,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.398071+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:17.398071+0000","last_peered":"2024-09-11T05:21:17.398071+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:17.398071+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15228928,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12840,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15946","reported_seq":16095,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.470790+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:17.470790+0000","last_peered":"2024-09-11T05:21:17.470790+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:17.470790+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14926441,"num_objects":3645,"num_object_clones":0,"num_object_copies":7290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12953,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243016321,"num_objects":59332,"num_object_clones":0,"num_object_copies":118664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222363,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9211,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":662822912,"data_stored":662745188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531432,"kb_used_data":233172,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93840408,"statfs":{"total":96636764160,"available":96092577792,"internally_reserved":0,"allocated":238768128,"data_stored":238498757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534164,"kb_used_data":181756,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93837676,"statfs":{"total":96636764160,"available":96089780224,"internally_reserved":0,"allocated":186118144,"data_stored":185785553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215209,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653360,"kb_used_data":234396,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93718480,"statfs":{"total":96636764160,"available":95967723520,"internally_reserved":0,"allocated":240021504,"data_stored":239752128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239349760,"data_stored":239346281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185544704,"data_stored":185478010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":237928448,"data_stored":237920897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16445", 'reported_seq': 16527, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.378740+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:17.378740+0000', 'last_peered': '2024-09-11T05:21:17.378740+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:17.378740+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15339520, 'num_objects': 3745, 'num_object_clones': 0, 'num_object_copies': 7490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16445, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 70}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16050", 'reported_seq': 20439, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.271072+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:17.271072+0000', 'last_peered': '2024-09-11T05:21:17.271072+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:17.271072+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15020056, 'num_objects': 3668, 'num_object_clones': 0, 'num_object_copies': 7336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3668, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14453, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2128, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2301}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'16460", 'reported_seq': 16597, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.446460+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:17.446460+0000', 'last_peered': '2024-09-11T05:21:17.446460+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:17.446460+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15327232, 'num_objects': 3742, 'num_object_clones': 0, 'num_object_copies': 7484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13625, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 166}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:18.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16253", 'reported_seq': 16389, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.267456+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:17.267456+0000', 'last_peered': '2024-09-11T05:21:17.267456+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:17.267456+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15228928, 'num_objects': 3718, 'num_object_clones': 0, 'num_object_copies': 7436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14321, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 193}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15961", 'reported_seq': 16132, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.397200+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:17.397200+0000', 'last_peered': '2024-09-11T05:21:17.397200+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:17.397200+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15200256, 'num_objects': 3711, 'num_object_clones': 0, 'num_object_copies': 7422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12714, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 269}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16322", 'reported_seq': 16445, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.240098+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:17.240098+0000', 'last_peered': '2024-09-11T05:21:17.240098+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:17.240098+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15200256, 'num_objects': 3711, 'num_object_clones': 0, 'num_object_copies': 7422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14351, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 189}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16442", 'reported_seq': 16579, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.412339+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:17.412339+0000', 'last_peered': '2024-09-11T05:21:17.412339+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:17.412339+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15163392, 'num_objects': 3702, 'num_object_clones': 0, 'num_object_copies': 7404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13475, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 79}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16011", 'reported_seq': 16150, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.453438+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:17.453438+0000', 'last_peered': '2024-09-11T05:21:17.453438+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:17.453438+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14921728, 'num_objects': 3643, 'num_object_clones': 0, 'num_object_copies': 7286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12877, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16082", 'reported_seq': 16222, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.398071+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:17.398071+0000', 'last_peered': '2024-09-11T05:21:17.398071+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:17.398071+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15228928, 'num_objects': 3718, 'num_object_clones': 0, 'num_object_copies': 7436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12840, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:18.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:18.307 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15946", 'reported_seq': 16095, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:17.470790+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:17.470790+0000', 'last_peered': '2024-09-11T05:21:17.470790+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:17.470790+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14926441, 'num_objects': 3645, 'num_object_clones': 0, 'num_object_copies': 7290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3645, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12953, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:18.307 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:19.433 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:19.433 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33301/118668 objects degraded (28.062%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":557,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59334,"data_bytes":243016340,"bytes_used":1760210944,"bytes_avail":288150081536,"bytes_total":289910292480,"degraded_objects":33301,"degraded_total":118668,"degraded_ratio":0.2806232514241413,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:19.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:19.480 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:20.250 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:20.250 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:20.319 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-11T05:21:19.588127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243016340,"num_objects":59334,"num_object_clones":0,"num_object_copies":118668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222365,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9212,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1718956,"kb_used_data":649324,"kb_used_omap":12,"kb_used_meta":1069555,"kb_avail":281396564,"statfs":{"total":289910292480,"available":288150081536,"internally_reserved":0,"allocated":664907776,"data_stored":664036438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1095224321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,6,11,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18661376,"num_objects":-4556,"num_object_clones":0,"num_object_copies":-9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005463"},"pg_stats":[{"pgid":"2.f","version":"47'16445","reported_seq":16527,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.378740+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:17.378740+0000","last_peered":"2024-09-11T05:21:17.378740+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:17.378740+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15339520,"num_objects":3745,"num_object_clones":0,"num_object_copies":7490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16445,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16219","reported_seq":16327,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.305089+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:17.305089+0000","last_peered":"2024-09-11T05:21:17.305089+0000","last_clean":"2024-09-11T05:21:17.305089+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:17.305089+0000","last_undegraded":"2024-09-11T05:21:17.305089+0000","last_fullsized":"2024-09-11T05:21:17.305089+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15077376,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14617,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16050","reported_seq":20439,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.271072+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:17.271072+0000","last_peered":"2024-09-11T05:21:17.271072+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:17.271072+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15020056,"num_objects":3668,"num_object_clones":0,"num_object_copies":7336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3668,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14453,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2128,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2301}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16334","reported_seq":16411,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.276468+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:17.276468+0000","last_peered":"2024-09-11T05:21:17.276468+0000","last_clean":"2024-09-11T05:21:17.276468+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:17.276468+0000","last_undegraded":"2024-09-11T05:21:17.276468+0000","last_fullsized":"2024-09-11T05:21:17.276468+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14852096,"num_objects":3626,"num_object_clones":0,"num_object_copies":7252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16424","reported_seq":16559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.466444+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:17.466444+0000","last_peered":"2024-09-11T05:21:17.466444+0000","last_clean":"2024-09-11T05:21:17.466444+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:17.466444+0000","last_undegraded":"2024-09-11T05:21:17.466444+0000","last_fullsized":"2024-09-11T05:21:17.466444+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15339520,"num_objects":3745,"num_object_clones":0,"num_object_copies":7490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13258,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16226","reported_seq":23577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:14.996369+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:14.996369+0000","last_peered":"2024-09-11T05:21:14.996369+0000","last_clean":"2024-09-11T05:21:14.996369+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:14.996369+0000","last_undegraded":"2024-09-11T05:21:14.996369+0000","last_fullsized":"2024-09-11T05:21:14.996369+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15601664,"num_objects":3809,"num_object_clones":0,"num_object_copies":7618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12770,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16460","reported_seq":16597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.446460+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:17.446460+0000","last_peered":"2024-09-11T05:21:17.446460+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:17.446460+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13625,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16150","reported_seq":16279,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.440929+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:17.440929+0000","last_peered":"2024-09-11T05:21:17.440929+0000","last_clean":"2024-09-11T05:21:17.440929+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:17.440929+0000","last_undegraded":"2024-09-11T05:21:17.440929+0000","last_fullsized":"2024-09-11T05:21:17.440929+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294464,"num_objects":3734,"num_object_clones":0,"num_object_copies":7468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13453,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16253","reported_seq":16389,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.267456+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:17.267456+0000","last_peered":"2024-09-11T05:21:17.267456+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:17.267456+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15228928,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14321,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15961","reported_seq":16132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.397200+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:17.397200+0000","last_peered":"2024-09-11T05:21:17.397200+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:17.397200+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15200256,"num_objects":3711,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12714,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16322","reported_seq":16445,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.240098+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:17.240098+0000","last_peered":"2024-09-11T05:21:17.240098+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:17.240098+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15200256,"num_objects":3711,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14351,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":189}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15974","reported_seq":23065,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:17.292175+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:17.292175+0000","last_peered":"2024-09-11T05:21:17.292175+0000","last_clean":"2024-09-11T05:21:17.292175+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:17.292175+0000","last_undegraded":"2024-09-11T05:21:17.292175+0000","last_fullsized":"2024-09-11T05:21:17.292175+0000","mapping_epoch":44,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15294464,"num_objects":3734,"num_object_clones":0,"num_object_copies":7468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13877,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16442","reported_seq":16579,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.412339+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:17.412339+0000","last_peered":"2024-09-11T05:21:17.412339+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:17.412339+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15163392,"num_objects":3702,"num_object_clones":0,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13475,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":79}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16011","reported_seq":16150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.453438+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:17.453438+0000","last_peered":"2024-09-11T05:21:17.453438+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:17.453438+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14921728,"num_objects":3643,"num_object_clones":0,"num_object_copies":7286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16082","reported_seq":16222,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.398071+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:17.398071+0000","last_peered":"2024-09-11T05:21:17.398071+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:17.398071+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15228928,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12840,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15946","reported_seq":16095,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:17.470790+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:17.470790+0000","last_peered":"2024-09-11T05:21:17.470790+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:17.470790+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14926441,"num_objects":3645,"num_object_clones":0,"num_object_copies":7290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12953,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243016321,"num_objects":59332,"num_object_clones":0,"num_object_copies":118664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222363,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9211,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":662822912,"data_stored":662745188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531432,"kb_used_data":233172,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93840408,"statfs":{"total":96636764160,"available":96092577792,"internally_reserved":0,"allocated":238768128,"data_stored":238498757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534164,"kb_used_data":181756,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93837676,"statfs":{"total":96636764160,"available":96089780224,"internally_reserved":0,"allocated":186118144,"data_stored":185785553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215209,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653360,"kb_used_data":234396,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93718480,"statfs":{"total":96636764160,"available":95967723520,"internally_reserved":0,"allocated":240021504,"data_stored":239752128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239349760,"data_stored":239346281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185544704,"data_stored":185478010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":237928448,"data_stored":237920897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:23.321 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:24.260 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-11T05:21:23.591874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233738900,"num_objects":57069,"num_object_clones":0,"num_object_copies":114138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57069,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224630,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9323,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160967,"ondisk_log_size":160967,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700724,"kb_used_data":631100,"kb_used_omap":12,"kb_used_meta":1069555,"kb_avail":281414796,"statfs":{"total":289910292480,"available":288168751104,"internally_reserved":0,"allocated":646246400,"data_stored":645375062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1095224321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,5,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27373568,"num_objects":-6683,"num_object_clones":0,"num_object_copies":-13366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008233"},"pg_stats":[{"pgid":"2.f","version":"47'16599","reported_seq":16683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.449618+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:22.449618+0000","last_peered":"2024-09-11T05:21:22.449618+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:22.449618+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14708736,"num_objects":3591,"num_object_clones":0,"num_object_copies":7182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16599,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-84}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16366","reported_seq":16474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.151502+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:22.151502+0000","last_peered":"2024-09-11T05:21:22.151502+0000","last_clean":"2024-09-11T05:21:22.151502+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:22.151502+0000","last_undegraded":"2024-09-11T05:21:22.151502+0000","last_fullsized":"2024-09-11T05:21:22.151502+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14764,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16178","reported_seq":20789,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.336307+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:22.336307+0000","last_peered":"2024-09-11T05:21:22.336307+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:22.336307+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14495768,"num_objects":3540,"num_object_clones":0,"num_object_copies":7080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14581,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2239,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16470","reported_seq":16547,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.337458+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:22.337458+0000","last_peered":"2024-09-11T05:21:22.337458+0000","last_clean":"2024-09-11T05:21:22.337458+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:22.337458+0000","last_undegraded":"2024-09-11T05:21:22.337458+0000","last_fullsized":"2024-09-11T05:21:22.337458+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14295040,"num_objects":3490,"num_object_clones":0,"num_object_copies":6980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16470,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16565","reported_seq":16700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.485865+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:22.485865+0000","last_peered":"2024-09-11T05:21:22.485865+0000","last_clean":"2024-09-11T05:21:22.485865+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:22.485865+0000","last_undegraded":"2024-09-11T05:21:22.485865+0000","last_fullsized":"2024-09-11T05:21:22.485865+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14761984,"num_objects":3604,"num_object_clones":0,"num_object_copies":7208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13399,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16366","reported_seq":23717,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:20.127734+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:20.127734+0000","last_peered":"2024-09-11T05:21:20.127734+0000","last_clean":"2024-09-11T05:21:20.127734+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:20.127734+0000","last_undegraded":"2024-09-11T05:21:20.127734+0000","last_fullsized":"2024-09-11T05:21:20.127734+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15028224,"num_objects":3669,"num_object_clones":0,"num_object_copies":7338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12910,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16580","reported_seq":16718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.463519+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:22.463519+0000","last_peered":"2024-09-11T05:21:22.463519+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:22.463519+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14835712,"num_objects":3622,"num_object_clones":0,"num_object_copies":7244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":46}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16315","reported_seq":16445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.433189+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:22.433189+0000","last_peered":"2024-09-11T05:21:22.433189+0000","last_clean":"2024-09-11T05:21:22.433189+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:22.433189+0000","last_undegraded":"2024-09-11T05:21:22.433189+0000","last_fullsized":"2024-09-11T05:21:22.433189+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14618624,"num_objects":3569,"num_object_clones":0,"num_object_copies":7138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13618,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16393","reported_seq":16530,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.274642+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:22.274642+0000","last_peered":"2024-09-11T05:21:22.274642+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:22.274642+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14655488,"num_objects":3578,"num_object_clones":0,"num_object_copies":7156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14461,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":53}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16112","reported_seq":16283,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.479550+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:22.479550+0000","last_peered":"2024-09-11T05:21:22.479550+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:22.479550+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14581760,"num_objects":3560,"num_object_clones":0,"num_object_copies":7120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12865,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16452","reported_seq":16575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.289098+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:22.289098+0000","last_peered":"2024-09-11T05:21:22.289098+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:22.289098+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14667776,"num_objects":3581,"num_object_clones":0,"num_object_copies":7162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14481,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":59}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16106","reported_seq":23197,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.312585+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:22.312585+0000","last_peered":"2024-09-11T05:21:22.312585+0000","last_clean":"2024-09-11T05:21:22.312585+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:22.312585+0000","last_undegraded":"2024-09-11T05:21:22.312585+0000","last_fullsized":"2024-09-11T05:21:22.312585+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753792,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14009,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16598","reported_seq":16735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.347269+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:22.347269+0000","last_peered":"2024-09-11T05:21:22.347269+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:22.347269+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13631,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-77}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16151","reported_seq":16290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.500211+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:22.500211+0000","last_peered":"2024-09-11T05:21:22.500211+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:22.500211+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13017,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16218","reported_seq":16356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.513557+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:22.513557+0000","last_peered":"2024-09-11T05:21:22.513557+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:22.513557+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14671872,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16095","reported_seq":16244,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.479636+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:22.479636+0000","last_peered":"2024-09-11T05:21:22.479636+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:22.479636+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14316137,"num_objects":3496,"num_object_clones":0,"num_object_copies":6992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13102,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233738881,"num_objects":57067,"num_object_clones":0,"num_object_copies":114134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57067,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224628,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9322,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":644222976,"data_stored":644145252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531124,"kb_used_data":228704,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93840716,"statfs":{"total":96636764160,"available":96092893184,"internally_reserved":0,"allocated":234192896,"data_stored":233923525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535748,"kb_used_data":175028,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93836092,"statfs":{"total":96636764160,"available":96088158208,"internally_reserved":0,"allocated":179228672,"data_stored":178896081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633852,"kb_used_data":227368,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93737988,"statfs":{"total":96636764160,"available":95987699712,"internally_reserved":0,"allocated":232824832,"data_stored":232555456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232239104,"data_stored":232235625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178642944,"data_stored":178576250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":233340928,"data_stored":233333377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:24.261 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:25.220 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:25.260 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-11T05:21:23.591874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233738900,"num_objects":57069,"num_object_clones":0,"num_object_copies":114138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57069,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224630,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9323,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160967,"ondisk_log_size":160967,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700724,"kb_used_data":631100,"kb_used_omap":12,"kb_used_meta":1069555,"kb_avail":281414796,"statfs":{"total":289910292480,"available":288168751104,"internally_reserved":0,"allocated":646246400,"data_stored":645375062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1095224321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,5,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27373568,"num_objects":-6683,"num_object_clones":0,"num_object_copies":-13366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008233"},"pg_stats":[{"pgid":"2.f","version":"47'16599","reported_seq":16683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.449618+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:22.449618+0000","last_peered":"2024-09-11T05:21:22.449618+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:22.449618+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14708736,"num_objects":3591,"num_object_clones":0,"num_object_copies":7182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16599,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-84}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16366","reported_seq":16474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.151502+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:22.151502+0000","last_peered":"2024-09-11T05:21:22.151502+0000","last_clean":"2024-09-11T05:21:22.151502+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:22.151502+0000","last_undegraded":"2024-09-11T05:21:22.151502+0000","last_fullsized":"2024-09-11T05:21:22.151502+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14764,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16178","reported_seq":20789,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.336307+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:22.336307+0000","last_peered":"2024-09-11T05:21:22.336307+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:22.336307+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14495768,"num_objects":3540,"num_object_clones":0,"num_object_copies":7080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14581,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2239,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16470","reported_seq":16547,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.337458+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:22.337458+0000","last_peered":"2024-09-11T05:21:22.337458+0000","last_clean":"2024-09-11T05:21:22.337458+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:22.337458+0000","last_undegraded":"2024-09-11T05:21:22.337458+0000","last_fullsized":"2024-09-11T05:21:22.337458+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14295040,"num_objects":3490,"num_object_clones":0,"num_object_copies":6980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16470,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16565","reported_seq":16700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.485865+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:22.485865+0000","last_peered":"2024-09-11T05:21:22.485865+0000","last_clean":"2024-09-11T05:21:22.485865+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:22.485865+0000","last_undegraded":"2024-09-11T05:21:22.485865+0000","last_fullsized":"2024-09-11T05:21:22.485865+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14761984,"num_objects":3604,"num_object_clones":0,"num_object_copies":7208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13399,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16366","reported_seq":23717,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:20.127734+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:20.127734+0000","last_peered":"2024-09-11T05:21:20.127734+0000","last_clean":"2024-09-11T05:21:20.127734+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:20.127734+0000","last_undegraded":"2024-09-11T05:21:20.127734+0000","last_fullsized":"2024-09-11T05:21:20.127734+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15028224,"num_objects":3669,"num_object_clones":0,"num_object_copies":7338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12910,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16580","reported_seq":16718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.463519+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:22.463519+0000","last_peered":"2024-09-11T05:21:22.463519+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:22.463519+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14835712,"num_objects":3622,"num_object_clones":0,"num_object_copies":7244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":46}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16315","reported_seq":16445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.433189+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:22.433189+0000","last_peered":"2024-09-11T05:21:22.433189+0000","last_clean":"2024-09-11T05:21:22.433189+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:22.433189+0000","last_undegraded":"2024-09-11T05:21:22.433189+0000","last_fullsized":"2024-09-11T05:21:22.433189+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14618624,"num_objects":3569,"num_object_clones":0,"num_object_copies":7138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13618,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16393","reported_seq":16530,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.274642+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:22.274642+0000","last_peered":"2024-09-11T05:21:22.274642+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:22.274642+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14655488,"num_objects":3578,"num_object_clones":0,"num_object_copies":7156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14461,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":53}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16112","reported_seq":16283,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.479550+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:22.479550+0000","last_peered":"2024-09-11T05:21:22.479550+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:22.479550+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14581760,"num_objects":3560,"num_object_clones":0,"num_object_copies":7120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12865,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16452","reported_seq":16575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.289098+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:22.289098+0000","last_peered":"2024-09-11T05:21:22.289098+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:22.289098+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14667776,"num_objects":3581,"num_object_clones":0,"num_object_copies":7162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14481,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":59}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16106","reported_seq":23197,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.312585+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:22.312585+0000","last_peered":"2024-09-11T05:21:22.312585+0000","last_clean":"2024-09-11T05:21:22.312585+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:22.312585+0000","last_undegraded":"2024-09-11T05:21:22.312585+0000","last_fullsized":"2024-09-11T05:21:22.312585+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753792,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14009,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16598","reported_seq":16735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.347269+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:22.347269+0000","last_peered":"2024-09-11T05:21:22.347269+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:22.347269+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13631,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-77}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16151","reported_seq":16290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.500211+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:22.500211+0000","last_peered":"2024-09-11T05:21:22.500211+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:22.500211+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13017,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16218","reported_seq":16356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.513557+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:22.513557+0000","last_peered":"2024-09-11T05:21:22.513557+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:22.513557+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14671872,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16095","reported_seq":16244,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.479636+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:22.479636+0000","last_peered":"2024-09-11T05:21:22.479636+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:22.479636+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14316137,"num_objects":3496,"num_object_clones":0,"num_object_copies":6992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13102,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233738881,"num_objects":57067,"num_object_clones":0,"num_object_copies":114134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57067,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224628,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9322,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":644222976,"data_stored":644145252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531124,"kb_used_data":228704,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93840716,"statfs":{"total":96636764160,"available":96092893184,"internally_reserved":0,"allocated":234192896,"data_stored":233923525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535748,"kb_used_data":175028,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93836092,"statfs":{"total":96636764160,"available":96088158208,"internally_reserved":0,"allocated":179228672,"data_stored":178896081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633852,"kb_used_data":227368,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93737988,"statfs":{"total":96636764160,"available":95987699712,"internally_reserved":0,"allocated":232824832,"data_stored":232555456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232239104,"data_stored":232235625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178642944,"data_stored":178576250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":233340928,"data_stored":233333377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:25.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:25.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16599", 'reported_seq': 16683, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.449618+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:22.449618+0000', 'last_peered': '2024-09-11T05:21:22.449618+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:22.449618+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14708736, 'num_objects': 3591, 'num_object_clones': 0, 'num_object_copies': 7182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16599, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -84}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:25.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:25.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16178", 'reported_seq': 20789, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.336307+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:22.336307+0000', 'last_peered': '2024-09-11T05:21:22.336307+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:22.336307+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14495768, 'num_objects': 3540, 'num_object_clones': 0, 'num_object_copies': 7080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1256, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3540, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14581, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2239, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'16580", 'reported_seq': 16718, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.463519+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:22.463519+0000', 'last_peered': '2024-09-11T05:21:22.463519+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:22.463519+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14835712, 'num_objects': 3622, 'num_object_clones': 0, 'num_object_copies': 7244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3622, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13745, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 46}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16393", 'reported_seq': 16530, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.274642+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:22.274642+0000', 'last_peered': '2024-09-11T05:21:22.274642+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:22.274642+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14655488, 'num_objects': 3578, 'num_object_clones': 0, 'num_object_copies': 7156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14461, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 53}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16112", 'reported_seq': 16283, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.479550+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:22.479550+0000', 'last_peered': '2024-09-11T05:21:22.479550+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:22.479550+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14581760, 'num_objects': 3560, 'num_object_clones': 0, 'num_object_copies': 7120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12865, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 118}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16452", 'reported_seq': 16575, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.289098+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:22.289098+0000', 'last_peered': '2024-09-11T05:21:22.289098+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:22.289098+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14667776, 'num_objects': 3581, 'num_object_clones': 0, 'num_object_copies': 7162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14481, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 59}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16598", 'reported_seq': 16735, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.347269+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:22.347269+0000', 'last_peered': '2024-09-11T05:21:22.347269+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:22.347269+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14524416, 'num_objects': 3546, 'num_object_clones': 0, 'num_object_copies': 7092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13631, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -77}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16151", 'reported_seq': 16290, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.500211+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:22.500211+0000', 'last_peered': '2024-09-11T05:21:22.500211+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:22.500211+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14348288, 'num_objects': 3503, 'num_object_clones': 0, 'num_object_copies': 7006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13017, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16218", 'reported_seq': 16356, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.513557+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:22.513557+0000', 'last_peered': '2024-09-11T05:21:22.513557+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:22.513557+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14671872, 'num_objects': 3582, 'num_object_clones': 0, 'num_object_copies': 7164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12976, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 38}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16095", 'reported_seq': 16244, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:22.479636+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:22.479636+0000', 'last_peered': '2024-09-11T05:21:22.479636+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:22.479636+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14316137, 'num_objects': 3496, 'num_object_clones': 0, 'num_object_copies': 6992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13102, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -19}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:25.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:26.378 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:26.378 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33190/113834 objects degraded (29.156%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":564,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":56917,"data_bytes":233116308,"bytes_used":1738735616,"bytes_avail":288171556864,"bytes_total":289910292480,"degraded_objects":33190,"degraded_total":113834,"degraded_ratio":0.29156491030799236,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":392},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:26.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:26.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:27.255 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:27.257 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:27.287 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-11T05:21:25.592402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233116308,"num_objects":56917,"num_object_clones":0,"num_object_copies":113834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224782,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9323,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1697984,"kb_used_data":624200,"kb_used_omap":12,"kb_used_meta":1073715,"kb_avail":281417536,"statfs":{"total":289910292480,"available":288171556864,"internally_reserved":0,"allocated":639180800,"data_stored":638309462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1099484161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,8,16,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19312640,"num_objects":-4715,"num_object_clones":0,"num_object_copies":-9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008330"},"pg_stats":[{"pgid":"2.f","version":"47'16599","reported_seq":16683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.449618+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:22.449618+0000","last_peered":"2024-09-11T05:21:22.449618+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:22.449618+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14708736,"num_objects":3591,"num_object_clones":0,"num_object_copies":7182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16599,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-84}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16366","reported_seq":16474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.151502+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:22.151502+0000","last_peered":"2024-09-11T05:21:22.151502+0000","last_clean":"2024-09-11T05:21:22.151502+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:22.151502+0000","last_undegraded":"2024-09-11T05:21:22.151502+0000","last_fullsized":"2024-09-11T05:21:22.151502+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14764,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16178","reported_seq":20789,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.336307+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:22.336307+0000","last_peered":"2024-09-11T05:21:22.336307+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:22.336307+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14495768,"num_objects":3540,"num_object_clones":0,"num_object_copies":7080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14581,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2239,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16470","reported_seq":16547,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.337458+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:22.337458+0000","last_peered":"2024-09-11T05:21:22.337458+0000","last_clean":"2024-09-11T05:21:22.337458+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:22.337458+0000","last_undegraded":"2024-09-11T05:21:22.337458+0000","last_fullsized":"2024-09-11T05:21:22.337458+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14295040,"num_objects":3490,"num_object_clones":0,"num_object_copies":6980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16470,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16565","reported_seq":16700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.485865+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:22.485865+0000","last_peered":"2024-09-11T05:21:22.485865+0000","last_clean":"2024-09-11T05:21:22.485865+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:22.485865+0000","last_undegraded":"2024-09-11T05:21:22.485865+0000","last_fullsized":"2024-09-11T05:21:22.485865+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14761984,"num_objects":3604,"num_object_clones":0,"num_object_copies":7208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13399,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16518","reported_seq":23867,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:25.229720+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:25.229720+0000","last_peered":"2024-09-11T05:21:25.229720+0000","last_clean":"2024-09-11T05:21:25.229720+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:25.229720+0000","last_undegraded":"2024-09-11T05:21:25.229720+0000","last_fullsized":"2024-09-11T05:21:25.229720+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14405632,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13062,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16580","reported_seq":16718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.463519+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:22.463519+0000","last_peered":"2024-09-11T05:21:22.463519+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:22.463519+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14835712,"num_objects":3622,"num_object_clones":0,"num_object_copies":7244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":46}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16315","reported_seq":16445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.433189+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:22.433189+0000","last_peered":"2024-09-11T05:21:22.433189+0000","last_clean":"2024-09-11T05:21:22.433189+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:22.433189+0000","last_undegraded":"2024-09-11T05:21:22.433189+0000","last_fullsized":"2024-09-11T05:21:22.433189+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14618624,"num_objects":3569,"num_object_clones":0,"num_object_copies":7138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13618,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16393","reported_seq":16530,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.274642+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:22.274642+0000","last_peered":"2024-09-11T05:21:22.274642+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:22.274642+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14655488,"num_objects":3578,"num_object_clones":0,"num_object_copies":7156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14461,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":53}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16112","reported_seq":16283,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.479550+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:22.479550+0000","last_peered":"2024-09-11T05:21:22.479550+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:22.479550+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14581760,"num_objects":3560,"num_object_clones":0,"num_object_copies":7120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12865,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16452","reported_seq":16575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.289098+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:22.289098+0000","last_peered":"2024-09-11T05:21:22.289098+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:22.289098+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14667776,"num_objects":3581,"num_object_clones":0,"num_object_copies":7162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14481,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":59}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16106","reported_seq":23197,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:22.312585+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:22.312585+0000","last_peered":"2024-09-11T05:21:22.312585+0000","last_clean":"2024-09-11T05:21:22.312585+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:22.312585+0000","last_undegraded":"2024-09-11T05:21:22.312585+0000","last_fullsized":"2024-09-11T05:21:22.312585+0000","mapping_epoch":44,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753792,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14009,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16598","reported_seq":16735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.347269+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:22.347269+0000","last_peered":"2024-09-11T05:21:22.347269+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:22.347269+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13631,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-77}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16151","reported_seq":16290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.500211+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:22.500211+0000","last_peered":"2024-09-11T05:21:22.500211+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:22.500211+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13017,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16218","reported_seq":16356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.513557+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:22.513557+0000","last_peered":"2024-09-11T05:21:22.513557+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:22.513557+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14671872,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16095","reported_seq":16244,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:22.479636+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:22.479636+0000","last_peered":"2024-09-11T05:21:22.479636+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:22.479636+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14316137,"num_objects":3496,"num_object_clones":0,"num_object_copies":6992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13102,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233116289,"num_objects":56915,"num_object_clones":0,"num_object_copies":113830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224780,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9322,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":637018112,"data_stored":636940388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531124,"kb_used_data":228704,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93840716,"statfs":{"total":96636764160,"available":96092893184,"internally_reserved":0,"allocated":234192896,"data_stored":233923525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533008,"kb_used_data":168128,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93838832,"statfs":{"total":96636764160,"available":96090963968,"internally_reserved":0,"allocated":172163072,"data_stored":171830481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633852,"kb_used_data":227368,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93737988,"statfs":{"total":96636764160,"available":95987699712,"internally_reserved":0,"allocated":232824832,"data_stored":232555456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232239104,"data_stored":232235625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171438080,"data_stored":171371386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":233340928,"data_stored":233333377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:30.289 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:31.128 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:31.130 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:31.206 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-11T05:21:29.593338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224400020,"num_objects":54789,"num_object_clones":0,"num_object_copies":109578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226910,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9431,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1694984,"kb_used_data":612888,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281420536,"statfs":{"total":289910292480,"available":288174628864,"internally_reserved":0,"allocated":627597312,"data_stored":626725974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1108003841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,1,11,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18616320,"num_objects":-4545,"num_object_clones":0,"num_object_copies":-9090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4545,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008328"},"pg_stats":[{"pgid":"2.f","version":"47'16745","reported_seq":16826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.478219+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:27.478219+0000","last_peered":"2024-09-11T05:21:27.478219+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:27.478219+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14110720,"num_objects":3445,"num_object_clones":0,"num_object_copies":6890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16745,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-230}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16518","reported_seq":16626,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.364779+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:27.364779+0000","last_peered":"2024-09-11T05:21:27.364779+0000","last_clean":"2024-09-11T05:21:27.364779+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:27.364779+0000","last_undegraded":"2024-09-11T05:21:27.364779+0000","last_fullsized":"2024-09-11T05:21:27.364779+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13852672,"num_objects":3382,"num_object_clones":0,"num_object_copies":6764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14916,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16300","reported_seq":21127,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.369683+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:27.369683+0000","last_peered":"2024-09-11T05:21:27.369683+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:27.369683+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13996056,"num_objects":3418,"num_object_clones":0,"num_object_copies":6836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3418,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14703,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2347,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16601","reported_seq":16678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.344596+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:27.344596+0000","last_peered":"2024-09-11T05:21:27.344596+0000","last_clean":"2024-09-11T05:21:27.344596+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:27.344596+0000","last_undegraded":"2024-09-11T05:21:27.344596+0000","last_fullsized":"2024-09-11T05:21:27.344596+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13758464,"num_objects":3359,"num_object_clones":0,"num_object_copies":6718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16601,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16726","reported_seq":16861,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.468354+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:27.468354+0000","last_peered":"2024-09-11T05:21:27.468354+0000","last_clean":"2024-09-11T05:21:27.468354+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:27.468354+0000","last_undegraded":"2024-09-11T05:21:27.468354+0000","last_fullsized":"2024-09-11T05:21:27.468354+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14102528,"num_objects":3443,"num_object_clones":0,"num_object_copies":6886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13560,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16518","reported_seq":23867,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:25.229720+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:25.229720+0000","last_peered":"2024-09-11T05:21:25.229720+0000","last_clean":"2024-09-11T05:21:25.229720+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:25.229720+0000","last_undegraded":"2024-09-11T05:21:25.229720+0000","last_fullsized":"2024-09-11T05:21:25.229720+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14405632,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13062,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16708","reported_seq":16846,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.496044+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:27.496044+0000","last_peered":"2024-09-11T05:21:27.496044+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:27.496044+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14311424,"num_objects":3494,"num_object_clones":0,"num_object_copies":6988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-82}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16439","reported_seq":16568,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.511460+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:27.511460+0000","last_peered":"2024-09-11T05:21:27.511460+0000","last_clean":"2024-09-11T05:21:27.511460+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:27.511460+0000","last_undegraded":"2024-09-11T05:21:27.511460+0000","last_fullsized":"2024-09-11T05:21:27.511460+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14110720,"num_objects":3445,"num_object_clones":0,"num_object_copies":6890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13742,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16542","reported_seq":16679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.247141+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:27.247141+0000","last_peered":"2024-09-11T05:21:27.247141+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:27.247141+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14045184,"num_objects":3429,"num_object_clones":0,"num_object_copies":6858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14610,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-96}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16259","reported_seq":16430,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.418627+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:27.418627+0000","last_peered":"2024-09-11T05:21:27.418627+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:27.418627+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13012,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16624","reported_seq":16742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.315072+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:27.315072+0000","last_peered":"2024-09-11T05:21:27.315072+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:27.315072+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13963264,"num_objects":3409,"num_object_clones":0,"num_object_copies":6818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14653,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16250","reported_seq":23341,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.327742+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:27.327742+0000","last_peered":"2024-09-11T05:21:27.327742+0000","last_clean":"2024-09-11T05:21:27.327742+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:27.327742+0000","last_undegraded":"2024-09-11T05:21:27.327742+0000","last_fullsized":"2024-09-11T05:21:27.327742+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14163968,"num_objects":3458,"num_object_clones":0,"num_object_copies":6916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14153,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16722","reported_seq":16859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.472923+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:27.472923+0000","last_peered":"2024-09-11T05:21:27.472923+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:27.472923+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14016512,"num_objects":3422,"num_object_clones":0,"num_object_copies":6844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13755,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16297","reported_seq":16436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.498536+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:27.498536+0000","last_peered":"2024-09-11T05:21:27.498536+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:27.498536+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13163,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16363","reported_seq":16504,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.468488+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:27.468488+0000","last_peered":"2024-09-11T05:21:27.468488+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:27.468488+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14077952,"num_objects":3437,"num_object_clones":0,"num_object_copies":6874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13121,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16232","reported_seq":16382,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.505615+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:27.505615+0000","last_peered":"2024-09-11T05:21:27.505615+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:27.505615+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13754985,"num_objects":3359,"num_object_clones":0,"num_object_copies":6718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13239,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224400001,"num_objects":54787,"num_object_clones":0,"num_object_copies":109574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54787,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226908,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9430,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":625405952,"data_stored":625328228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160644,"ondisk_log_size":160644,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530712,"kb_used_data":224136,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93841128,"statfs":{"total":96636764160,"available":96093315072,"internally_reserved":0,"allocated":229515264,"data_stored":229245893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533008,"kb_used_data":168128,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93838832,"statfs":{"total":96636764160,"available":96090963968,"internally_reserved":0,"allocated":172163072,"data_stored":171830481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631264,"kb_used_data":220624,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93740576,"statfs":{"total":96636764160,"available":95990349824,"internally_reserved":0,"allocated":225918976,"data_stored":225649600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,0,3,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":225333248,"data_stored":225329769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171438080,"data_stored":171371386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228634624,"data_stored":228627073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:31.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:32.038 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:32.039 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:32.088 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-11T05:21:31.596223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223871636,"num_objects":54660,"num_object_clones":0,"num_object_copies":109320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227039,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9431,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1679876,"kb_used_data":606096,"kb_used_omap":12,"kb_used_meta":1073715,"kb_avail":281435644,"statfs":{"total":289910292480,"available":288190099456,"internally_reserved":0,"allocated":620642304,"data_stored":619770966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1099484161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,0,3,11,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19144704,"num_objects":-4674,"num_object_clones":0,"num_object_copies":-9348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4674,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008096"},"pg_stats":[{"pgid":"2.f","version":"47'16745","reported_seq":16826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.478219+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:27.478219+0000","last_peered":"2024-09-11T05:21:27.478219+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:27.478219+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14110720,"num_objects":3445,"num_object_clones":0,"num_object_copies":6890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16745,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-230}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16518","reported_seq":16626,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.364779+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:27.364779+0000","last_peered":"2024-09-11T05:21:27.364779+0000","last_clean":"2024-09-11T05:21:27.364779+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:27.364779+0000","last_undegraded":"2024-09-11T05:21:27.364779+0000","last_fullsized":"2024-09-11T05:21:27.364779+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13852672,"num_objects":3382,"num_object_clones":0,"num_object_copies":6764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14916,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16300","reported_seq":21127,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.369683+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:27.369683+0000","last_peered":"2024-09-11T05:21:27.369683+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:27.369683+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13996056,"num_objects":3418,"num_object_clones":0,"num_object_copies":6836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3418,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14703,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2347,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16601","reported_seq":16678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.344596+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:27.344596+0000","last_peered":"2024-09-11T05:21:27.344596+0000","last_clean":"2024-09-11T05:21:27.344596+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:27.344596+0000","last_undegraded":"2024-09-11T05:21:27.344596+0000","last_fullsized":"2024-09-11T05:21:27.344596+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13758464,"num_objects":3359,"num_object_clones":0,"num_object_copies":6718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16601,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16726","reported_seq":16861,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.468354+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:27.468354+0000","last_peered":"2024-09-11T05:21:27.468354+0000","last_clean":"2024-09-11T05:21:27.468354+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:27.468354+0000","last_undegraded":"2024-09-11T05:21:27.468354+0000","last_fullsized":"2024-09-11T05:21:27.468354+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14102528,"num_objects":3443,"num_object_clones":0,"num_object_copies":6886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13560,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16647","reported_seq":23998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:30.284493+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:30.284493+0000","last_peered":"2024-09-11T05:21:30.284493+0000","last_clean":"2024-09-11T05:21:30.284493+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:30.284493+0000","last_undegraded":"2024-09-11T05:21:30.284493+0000","last_fullsized":"2024-09-11T05:21:30.284493+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13877248,"num_objects":3388,"num_object_clones":0,"num_object_copies":6776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13191,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16708","reported_seq":16846,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.496044+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:27.496044+0000","last_peered":"2024-09-11T05:21:27.496044+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:27.496044+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14311424,"num_objects":3494,"num_object_clones":0,"num_object_copies":6988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-82}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16439","reported_seq":16568,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.511460+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:27.511460+0000","last_peered":"2024-09-11T05:21:27.511460+0000","last_clean":"2024-09-11T05:21:27.511460+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:27.511460+0000","last_undegraded":"2024-09-11T05:21:27.511460+0000","last_fullsized":"2024-09-11T05:21:27.511460+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14110720,"num_objects":3445,"num_object_clones":0,"num_object_copies":6890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13742,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16542","reported_seq":16679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.247141+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:27.247141+0000","last_peered":"2024-09-11T05:21:27.247141+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:27.247141+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14045184,"num_objects":3429,"num_object_clones":0,"num_object_copies":6858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14610,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-96}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16259","reported_seq":16430,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.418627+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:27.418627+0000","last_peered":"2024-09-11T05:21:27.418627+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:27.418627+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13012,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16624","reported_seq":16742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.315072+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:27.315072+0000","last_peered":"2024-09-11T05:21:27.315072+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:27.315072+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13963264,"num_objects":3409,"num_object_clones":0,"num_object_copies":6818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14653,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16250","reported_seq":23341,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:27.327742+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:27.327742+0000","last_peered":"2024-09-11T05:21:27.327742+0000","last_clean":"2024-09-11T05:21:27.327742+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:27.327742+0000","last_undegraded":"2024-09-11T05:21:27.327742+0000","last_fullsized":"2024-09-11T05:21:27.327742+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14163968,"num_objects":3458,"num_object_clones":0,"num_object_copies":6916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14153,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16722","reported_seq":16859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.472923+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:27.472923+0000","last_peered":"2024-09-11T05:21:27.472923+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:27.472923+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14016512,"num_objects":3422,"num_object_clones":0,"num_object_copies":6844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13755,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16297","reported_seq":16436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.498536+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:27.498536+0000","last_peered":"2024-09-11T05:21:27.498536+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:27.498536+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13163,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16363","reported_seq":16504,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.468488+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:27.468488+0000","last_peered":"2024-09-11T05:21:27.468488+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:27.468488+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14077952,"num_objects":3437,"num_object_clones":0,"num_object_copies":6874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13121,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16232","reported_seq":16382,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:27.505615+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:27.505615+0000","last_peered":"2024-09-11T05:21:27.505615+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:27.505615+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13754985,"num_objects":3359,"num_object_clones":0,"num_object_copies":6718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13239,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":223871617,"num_objects":54658,"num_object_clones":0,"num_object_copies":109316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227037,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9430,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":618237952,"data_stored":618160228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530712,"kb_used_data":224136,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93841128,"statfs":{"total":96636764160,"available":96093315072,"internally_reserved":0,"allocated":229515264,"data_stored":229245893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313913003},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517900,"kb_used_data":161336,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93853940,"statfs":{"total":96636764160,"available":96106434560,"internally_reserved":0,"allocated":165208064,"data_stored":164875473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631264,"kb_used_data":220624,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93740576,"statfs":{"total":96636764160,"available":95990349824,"internally_reserved":0,"allocated":225918976,"data_stored":225649600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,0,3,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":225333248,"data_stored":225329769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":164270080,"data_stored":164203386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228634624,"data_stored":228627073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:32.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16745", 'reported_seq': 16826, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.478219+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:27.478219+0000', 'last_peered': '2024-09-11T05:21:27.478219+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:27.478219+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14110720, 'num_objects': 3445, 'num_object_clones': 0, 'num_object_copies': 6890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16745, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -230}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16300", 'reported_seq': 21127, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.369683+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:27.369683+0000', 'last_peered': '2024-09-11T05:21:27.369683+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:27.369683+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13996056, 'num_objects': 3418, 'num_object_clones': 0, 'num_object_copies': 6836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1148, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3418, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14703, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2347, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2270}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'16708", 'reported_seq': 16846, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.496044+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:27.496044+0000', 'last_peered': '2024-09-11T05:21:27.496044+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:27.496044+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14311424, 'num_objects': 3494, 'num_object_clones': 0, 'num_object_copies': 6988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13873, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -82}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16542", 'reported_seq': 16679, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.247141+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:27.247141+0000', 'last_peered': '2024-09-11T05:21:27.247141+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:27.247141+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14045184, 'num_objects': 3429, 'num_object_clones': 0, 'num_object_copies': 6858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14610, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -96}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:32.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16259", 'reported_seq': 16430, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.418627+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:27.418627+0000', 'last_peered': '2024-09-11T05:21:27.418627+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:27.418627+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13979648, 'num_objects': 3413, 'num_object_clones': 0, 'num_object_copies': 6826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13012, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -29}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16624", 'reported_seq': 16742, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.315072+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:27.315072+0000', 'last_peered': '2024-09-11T05:21:27.315072+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:27.315072+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13963264, 'num_objects': 3409, 'num_object_clones': 0, 'num_object_copies': 6818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14653, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -113}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16722", 'reported_seq': 16859, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.472923+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:27.472923+0000', 'last_peered': '2024-09-11T05:21:27.472923+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:27.472923+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14016512, 'num_objects': 3422, 'num_object_clones': 0, 'num_object_copies': 6844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3422, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13755, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16297", 'reported_seq': 16436, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.498536+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:27.498536+0000', 'last_peered': '2024-09-11T05:21:27.498536+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:27.498536+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13750272, 'num_objects': 3357, 'num_object_clones': 0, 'num_object_copies': 6714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13163, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16363", 'reported_seq': 16504, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.468488+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:27.468488+0000', 'last_peered': '2024-09-11T05:21:27.468488+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:27.468488+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14077952, 'num_objects': 3437, 'num_object_clones': 0, 'num_object_copies': 6874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13121, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:32.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16232", 'reported_seq': 16382, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:27.505615+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:27.505615+0000', 'last_peered': '2024-09-11T05:21:27.505615+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:27.505615+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13754985, 'num_objects': 3359, 'num_object_clones': 0, 'num_object_copies': 6718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13239, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -156}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:32.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:33.063 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:33.063 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33082/109320 objects degraded (30.262%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":570,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":54660,"data_bytes":223871636,"bytes_used":1720193024,"bytes_avail":288190099456,"bytes_total":289910292480,"degraded_objects":33082,"degraded_total":109320,"degraded_ratio":0.30261617270398827,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:33.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:33.106 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:33.787 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:33.787 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:33.853 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-11T05:21:33.600115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":215106196,"num_objects":52520,"num_object_clones":0,"num_object_copies":105040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229179,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9553,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1681108,"kb_used_data":594828,"kb_used_omap":12,"kb_used_meta":1086195,"kb_avail":281434412,"statfs":{"total":289910292480,"available":288188837888,"internally_reserved":0,"allocated":609103872,"data_stored":608232534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1112263681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,15,12,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27336704,"num_objects":-6674,"num_object_clones":0,"num_object_copies":-13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6674,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008598"},"pg_stats":[{"pgid":"2.f","version":"47'16876","reported_seq":16960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.283264+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:32.283264+0000","last_peered":"2024-09-11T05:21:32.283264+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:32.283264+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16876,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-361}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16671","reported_seq":16779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.425439+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:32.425439+0000","last_peered":"2024-09-11T05:21:32.425439+0000","last_clean":"2024-09-11T05:21:32.425439+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:32.425439+0000","last_undegraded":"2024-09-11T05:21:32.425439+0000","last_fullsized":"2024-09-11T05:21:32.425439+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13225984,"num_objects":3229,"num_object_clones":0,"num_object_copies":6458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15069,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16447","reported_seq":21518,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.438886+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:32.438886+0000","last_peered":"2024-09-11T05:21:32.438886+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:32.438886+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13393944,"num_objects":3271,"num_object_clones":0,"num_object_copies":6542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14850,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2469,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16732","reported_seq":16807,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.321625+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:32.321625+0000","last_peered":"2024-09-11T05:21:32.321625+0000","last_clean":"2024-09-11T05:21:32.321625+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:32.321625+0000","last_undegraded":"2024-09-11T05:21:32.321625+0000","last_fullsized":"2024-09-11T05:21:32.321625+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13221888,"num_objects":3228,"num_object_clones":0,"num_object_copies":6456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16732,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16861","reported_seq":16995,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.520495+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:32.520495+0000","last_peered":"2024-09-11T05:21:32.520495+0000","last_clean":"2024-09-11T05:21:32.520495+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:32.520495+0000","last_undegraded":"2024-09-11T05:21:32.520495+0000","last_fullsized":"2024-09-11T05:21:32.520495+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16647","reported_seq":23998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:30.284493+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:30.284493+0000","last_peered":"2024-09-11T05:21:30.284493+0000","last_clean":"2024-09-11T05:21:30.284493+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:30.284493+0000","last_undegraded":"2024-09-11T05:21:30.284493+0000","last_fullsized":"2024-09-11T05:21:30.284493+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13877248,"num_objects":3388,"num_object_clones":0,"num_object_copies":6776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13191,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16871","reported_seq":17009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.497415+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:32.497415+0000","last_peered":"2024-09-11T05:21:32.497415+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:32.497415+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13643776,"num_objects":3331,"num_object_clones":0,"num_object_copies":6662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14036,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16570","reported_seq":16700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.463331+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:32.463331+0000","last_peered":"2024-09-11T05:21:32.463331+0000","last_clean":"2024-09-11T05:21:32.463331+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:32.463331+0000","last_undegraded":"2024-09-11T05:21:32.463331+0000","last_fullsized":"2024-09-11T05:21:32.463331+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16675","reported_seq":16811,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.264991+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:32.264991+0000","last_peered":"2024-09-11T05:21:32.264991+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:32.264991+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14743,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-229}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16414","reported_seq":16584,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.530988+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:32.530988+0000","last_peered":"2024-09-11T05:21:32.530988+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:32.530988+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13344768,"num_objects":3258,"num_object_clones":0,"num_object_copies":6516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13167,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16770","reported_seq":16893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.334434+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:32.334434+0000","last_peered":"2024-09-11T05:21:32.334434+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:32.334434+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13365248,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14799,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-259}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16401","reported_seq":23491,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.380367+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:32.380367+0000","last_peered":"2024-09-11T05:21:32.380367+0000","last_clean":"2024-09-11T05:21:32.380367+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:32.380367+0000","last_undegraded":"2024-09-11T05:21:32.380367+0000","last_fullsized":"2024-09-11T05:21:32.380367+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13545472,"num_objects":3307,"num_object_clones":0,"num_object_copies":6614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14304,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.490716+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:32.490716+0000","last_peered":"2024-09-11T05:21:32.490716+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:32.490716+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13430784,"num_objects":3279,"num_object_clones":0,"num_object_copies":6558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13898,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16431","reported_seq":16570,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.514376+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:32.514376+0000","last_peered":"2024-09-11T05:21:32.514376+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:32.514376+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201408,"num_objects":3223,"num_object_clones":0,"num_object_copies":6446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13297,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16492","reported_seq":16633,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.540647+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:32.540647+0000","last_peered":"2024-09-11T05:21:32.540647+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:32.540647+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13250,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16390","reported_seq":16540,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.463149+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:32.463149+0000","last_peered":"2024-09-11T05:21:32.463149+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:32.463149+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13107817,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13397,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":215106177,"num_objects":52518,"num_object_clones":0,"num_object_copies":105036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229177,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9552,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606609408,"data_stored":606531684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530352,"kb_used_data":219608,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93841488,"statfs":{"total":96636764160,"available":96093683712,"internally_reserved":0,"allocated":224878592,"data_stored":224609221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,9,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517900,"kb_used_data":161336,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93853940,"statfs":{"total":96636764160,"available":96106434560,"internally_reserved":0,"allocated":165208064,"data_stored":164875473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215212,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632856,"kb_used_data":213884,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93738984,"statfs":{"total":96636764160,"available":95988719616,"internally_reserved":0,"allocated":219017216,"data_stored":218747840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,1,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":218370048,"data_stored":218366569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":164270080,"data_stored":164203386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223969280,"data_stored":223961729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:36.855 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:37.560 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:37.561 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:37.629 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-11T05:21:35.600618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":214401684,"num_objects":52348,"num_object_clones":0,"num_object_copies":104696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229351,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9553,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1678608,"kb_used_data":588172,"kb_used_omap":12,"kb_used_meta":1090355,"kb_avail":281436912,"statfs":{"total":289910292480,"available":288191397888,"internally_reserved":0,"allocated":602288128,"data_stored":601416790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1116523521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,11,10,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19337216,"num_objects":-4721,"num_object_clones":0,"num_object_copies":-9442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4721,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008743"},"pg_stats":[{"pgid":"2.f","version":"47'16876","reported_seq":16960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.283264+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:32.283264+0000","last_peered":"2024-09-11T05:21:32.283264+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:32.283264+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16876,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-361}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16671","reported_seq":16779,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.425439+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:32.425439+0000","last_peered":"2024-09-11T05:21:32.425439+0000","last_clean":"2024-09-11T05:21:32.425439+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:32.425439+0000","last_undegraded":"2024-09-11T05:21:32.425439+0000","last_fullsized":"2024-09-11T05:21:32.425439+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13225984,"num_objects":3229,"num_object_clones":0,"num_object_copies":6458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15069,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16447","reported_seq":21518,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.438886+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:32.438886+0000","last_peered":"2024-09-11T05:21:32.438886+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:32.438886+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13393944,"num_objects":3271,"num_object_clones":0,"num_object_copies":6542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14850,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2469,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16732","reported_seq":16807,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.321625+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:32.321625+0000","last_peered":"2024-09-11T05:21:32.321625+0000","last_clean":"2024-09-11T05:21:32.321625+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:32.321625+0000","last_undegraded":"2024-09-11T05:21:32.321625+0000","last_fullsized":"2024-09-11T05:21:32.321625+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13221888,"num_objects":3228,"num_object_clones":0,"num_object_copies":6456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16732,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16861","reported_seq":16995,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.520495+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:32.520495+0000","last_peered":"2024-09-11T05:21:32.520495+0000","last_clean":"2024-09-11T05:21:32.520495+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:32.520495+0000","last_undegraded":"2024-09-11T05:21:32.520495+0000","last_fullsized":"2024-09-11T05:21:32.520495+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16819","reported_seq":24168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:35.396738+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:35.396738+0000","last_peered":"2024-09-11T05:21:35.396738+0000","last_clean":"2024-09-11T05:21:35.396738+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:35.396738+0000","last_undegraded":"2024-09-11T05:21:35.396738+0000","last_fullsized":"2024-09-11T05:21:35.396738+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":3216,"num_object_clones":0,"num_object_copies":6432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13363,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16871","reported_seq":17009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.497415+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:32.497415+0000","last_peered":"2024-09-11T05:21:32.497415+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:32.497415+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13643776,"num_objects":3331,"num_object_clones":0,"num_object_copies":6662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14036,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16570","reported_seq":16700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.463331+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:32.463331+0000","last_peered":"2024-09-11T05:21:32.463331+0000","last_clean":"2024-09-11T05:21:32.463331+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:32.463331+0000","last_undegraded":"2024-09-11T05:21:32.463331+0000","last_fullsized":"2024-09-11T05:21:32.463331+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16675","reported_seq":16811,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.264991+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:32.264991+0000","last_peered":"2024-09-11T05:21:32.264991+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:32.264991+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14743,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-229}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.704927+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:25.704927+0000","last_peered":"2024-09-11T05:19:25.704927+0000","last_clean":"2024-09-11T05:19:25.704927+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:25.704927+0000","last_undegraded":"2024-09-11T05:19:25.704927+0000","last_fullsized":"2024-09-11T05:19:25.704927+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16414","reported_seq":16584,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.530988+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:32.530988+0000","last_peered":"2024-09-11T05:21:32.530988+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:32.530988+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13344768,"num_objects":3258,"num_object_clones":0,"num_object_copies":6516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13167,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16770","reported_seq":16893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.334434+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:32.334434+0000","last_peered":"2024-09-11T05:21:32.334434+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:32.334434+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13365248,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14799,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-259}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16401","reported_seq":23491,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.380367+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:32.380367+0000","last_peered":"2024-09-11T05:21:32.380367+0000","last_clean":"2024-09-11T05:21:32.380367+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:32.380367+0000","last_undegraded":"2024-09-11T05:21:32.380367+0000","last_fullsized":"2024-09-11T05:21:32.380367+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13545472,"num_objects":3307,"num_object_clones":0,"num_object_copies":6614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14304,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.490716+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:32.490716+0000","last_peered":"2024-09-11T05:21:32.490716+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:32.490716+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13430784,"num_objects":3279,"num_object_clones":0,"num_object_copies":6558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13898,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16431","reported_seq":16570,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.514376+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:32.514376+0000","last_peered":"2024-09-11T05:21:32.514376+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:32.514376+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201408,"num_objects":3223,"num_object_clones":0,"num_object_copies":6446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13297,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707024+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:25.707024+0000","last_peered":"2024-09-11T05:19:25.707024+0000","last_clean":"2024-09-11T05:19:25.707024+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:25.707024+0000","last_undegraded":"2024-09-11T05:19:25.707024+0000","last_fullsized":"2024-09-11T05:19:25.707024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16492","reported_seq":16633,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.540647+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:32.540647+0000","last_peered":"2024-09-11T05:21:32.540647+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:32.540647+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13250,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16390","reported_seq":16540,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.463149+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:32.463149+0000","last_peered":"2024-09-11T05:21:32.463149+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:32.463149+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13107817,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13397,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":214401665,"num_objects":52346,"num_object_clones":0,"num_object_copies":104692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229349,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9552,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":599609344,"data_stored":599531620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160885,"ondisk_log_size":160885,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530352,"kb_used_data":219608,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93841488,"statfs":{"total":96636764160,"available":96093683712,"internally_reserved":0,"allocated":224878592,"data_stored":224609221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318172843},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,9,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515400,"kb_used_data":154680,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93856440,"statfs":{"total":96636764160,"available":96108994560,"internally_reserved":0,"allocated":158392320,"data_stored":158059729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215212,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632856,"kb_used_data":213884,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93738984,"statfs":{"total":96636764160,"available":95988719616,"internally_reserved":0,"allocated":219017216,"data_stored":218747840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,1,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":218370048,"data_stored":218366569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157270016,"data_stored":157203322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223969280,"data_stored":223961729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:37.631 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:38.342 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:38.345 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:38.374 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-11T05:21:37.601063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":210850452,"num_objects":51481,"num_object_clones":0,"num_object_copies":102962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230218,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9663,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1661512,"kb_used_data":583544,"kb_used_omap":12,"kb_used_meta":1077875,"kb_avail":281454008,"statfs":{"total":289910292480,"available":288208904192,"internally_reserved":0,"allocated":597549056,"data_stored":596677718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1103744001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,2,6,11,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22265856,"num_objects":-5436,"num_object_clones":0,"num_object_copies":-10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5436,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008660"},"pg_stats":[{"pgid":"2.f","version":"47'16876","reported_seq":16960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.283264+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:32.283264+0000","last_peered":"2024-09-11T05:21:32.283264+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:32.283264+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16876,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-361}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16814","reported_seq":16922,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.438021+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:37.438021+0000","last_peered":"2024-09-11T05:21:37.438021+0000","last_clean":"2024-09-11T05:21:37.438021+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:37.438021+0000","last_undegraded":"2024-09-11T05:21:37.438021+0000","last_fullsized":"2024-09-11T05:21:37.438021+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12640256,"num_objects":3086,"num_object_clones":0,"num_object_copies":6172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15212,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16604","reported_seq":21893,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.437259+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:37.437259+0000","last_peered":"2024-09-11T05:21:37.437259+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:37.437259+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12750872,"num_objects":3114,"num_object_clones":0,"num_object_copies":6228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15007,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2579,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2198}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16864","reported_seq":16941,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.332454+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:37.332454+0000","last_peered":"2024-09-11T05:21:37.332454+0000","last_clean":"2024-09-11T05:21:37.332454+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:37.332454+0000","last_undegraded":"2024-09-11T05:21:37.332454+0000","last_fullsized":"2024-09-11T05:21:37.332454+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12681216,"num_objects":3096,"num_object_clones":0,"num_object_copies":6192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16864,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16861","reported_seq":16995,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.520495+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:32.520495+0000","last_peered":"2024-09-11T05:21:32.520495+0000","last_clean":"2024-09-11T05:21:32.520495+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:32.520495+0000","last_undegraded":"2024-09-11T05:21:32.520495+0000","last_fullsized":"2024-09-11T05:21:32.520495+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16819","reported_seq":24168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:35.396738+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:35.396738+0000","last_peered":"2024-09-11T05:21:35.396738+0000","last_clean":"2024-09-11T05:21:35.396738+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:35.396738+0000","last_undegraded":"2024-09-11T05:21:35.396738+0000","last_fullsized":"2024-09-11T05:21:35.396738+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":3216,"num_object_clones":0,"num_object_copies":6432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13363,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'16871","reported_seq":17009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.497415+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:32.497415+0000","last_peered":"2024-09-11T05:21:32.497415+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:32.497415+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13643776,"num_objects":3331,"num_object_clones":0,"num_object_copies":6662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14036,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16570","reported_seq":16700,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:32.463331+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:32.463331+0000","last_peered":"2024-09-11T05:21:32.463331+0000","last_clean":"2024-09-11T05:21:32.463331+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:32.463331+0000","last_undegraded":"2024-09-11T05:21:32.463331+0000","last_fullsized":"2024-09-11T05:21:32.463331+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16836","reported_seq":16973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.344305+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:37.344305+0000","last_peered":"2024-09-11T05:21:37.344305+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:37.344305+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12840960,"num_objects":3135,"num_object_clones":0,"num_object_copies":6270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-390}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16414","reported_seq":16584,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.530988+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:32.530988+0000","last_peered":"2024-09-11T05:21:32.530988+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:32.530988+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13344768,"num_objects":3258,"num_object_clones":0,"num_object_copies":6516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13167,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16902","reported_seq":17026,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.420547+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:37.420547+0000","last_peered":"2024-09-11T05:21:37.420547+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:37.420547+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12824576,"num_objects":3131,"num_object_clones":0,"num_object_copies":6262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14931,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-391}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16543","reported_seq":23632,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.462575+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:37.462575+0000","last_peered":"2024-09-11T05:21:37.462575+0000","last_clean":"2024-09-11T05:21:37.462575+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:37.462575+0000","last_undegraded":"2024-09-11T05:21:37.462575+0000","last_fullsized":"2024-09-11T05:21:37.462575+0000","mapping_epoch":44,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12963840,"num_objects":3165,"num_object_clones":0,"num_object_copies":6330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14446,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.490716+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:32.490716+0000","last_peered":"2024-09-11T05:21:32.490716+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:32.490716+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13430784,"num_objects":3279,"num_object_clones":0,"num_object_copies":6558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13898,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-344}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16431","reported_seq":16570,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.514376+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:32.514376+0000","last_peered":"2024-09-11T05:21:32.514376+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:32.514376+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201408,"num_objects":3223,"num_object_clones":0,"num_object_copies":6446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13297,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16492","reported_seq":16633,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.540647+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:32.540647+0000","last_peered":"2024-09-11T05:21:32.540647+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:32.540647+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13250,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16390","reported_seq":16540,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:32.463149+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:32.463149+0000","last_peered":"2024-09-11T05:21:32.463149+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:32.463149+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13107817,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13397,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":210850433,"num_objects":51479,"num_object_clones":0,"num_object_copies":102958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230216,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9662,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":594837504,"data_stored":594759780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513256,"kb_used_data":214980,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93858584,"statfs":{"total":96636764160,"available":96111190016,"internally_reserved":0,"allocated":220139520,"data_stored":219870149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515400,"kb_used_data":154680,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93856440,"statfs":{"total":96636764160,"available":96108994560,"internally_reserved":0,"allocated":158392320,"data_stored":158059729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215212,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632856,"kb_used_data":213884,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93738984,"statfs":{"total":96636764160,"available":95988719616,"internally_reserved":0,"allocated":219017216,"data_stored":218747840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,1,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":218370048,"data_stored":218366569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157270016,"data_stored":157203322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":219197440,"data_stored":219189889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:38.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:38.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16876", 'reported_seq': 16960, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:32.283264+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:32.283264+0000', 'last_peered': '2024-09-11T05:21:32.283264+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:32.283264+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13574144, 'num_objects': 3314, 'num_object_clones': 0, 'num_object_copies': 6628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16876, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -361}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:38.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:38.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16604", 'reported_seq': 21893, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:37.437259+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:37.437259+0000', 'last_peered': '2024-09-11T05:21:37.437259+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:37.437259+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12750872, 'num_objects': 3114, 'num_object_clones': 0, 'num_object_copies': 6228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 916, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3114, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15007, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2579, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2198}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'16871", 'reported_seq': 17009, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:32.497415+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:32.497415+0000', 'last_peered': '2024-09-11T05:21:32.497415+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:32.497415+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13643776, 'num_objects': 3331, 'num_object_clones': 0, 'num_object_copies': 6662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14036, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16836", 'reported_seq': 16973, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:37.344305+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:37.344305+0000', 'last_peered': '2024-09-11T05:21:37.344305+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:37.344305+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12840960, 'num_objects': 3135, 'num_object_clones': 0, 'num_object_copies': 6270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14904, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -390}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16414", 'reported_seq': 16584, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:32.530988+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:32.530988+0000', 'last_peered': '2024-09-11T05:21:32.530988+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:32.530988+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13344768, 'num_objects': 3258, 'num_object_clones': 0, 'num_object_copies': 6516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13167, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -184}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:38.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16902", 'reported_seq': 17026, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:37.420547+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:37.420547+0000', 'last_peered': '2024-09-11T05:21:37.420547+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:37.420547+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12824576, 'num_objects': 3131, 'num_object_clones': 0, 'num_object_copies': 6262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14931, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -391}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16865", 'reported_seq': 17002, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:32.490716+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:32.490716+0000', 'last_peered': '2024-09-11T05:21:32.490716+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:32.490716+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13430784, 'num_objects': 3279, 'num_object_clones': 0, 'num_object_copies': 6558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13898, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -344}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16431", 'reported_seq': 16570, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:32.514376+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:32.514376+0000', 'last_peered': '2024-09-11T05:21:32.514376+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:32.514376+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13201408, 'num_objects': 3223, 'num_object_clones': 0, 'num_object_copies': 6446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3223, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13297, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -272}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16492", 'reported_seq': 16633, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:32.540647+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:32.540647+0000', 'last_peered': '2024-09-11T05:21:32.540647+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:32.540647+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13549568, 'num_objects': 3308, 'num_object_clones': 0, 'num_object_copies': 6616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13250, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:38.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16390", 'reported_seq': 16540, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:32.463149+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:32.463149+0000', 'last_peered': '2024-09-11T05:21:32.463149+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:32.463149+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13107817, 'num_objects': 3201, 'num_object_clones': 0, 'num_object_copies': 6402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13397, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:38.378 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:39.262 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:39.262 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32850/102962 objects degraded (31.905%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":577,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":51481,"data_bytes":210850452,"bytes_used":1701388288,"bytes_avail":288208904192,"bytes_total":289910292480,"degraded_objects":32850,"degraded_total":102962,"degraded_ratio":0.31904974650842061,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":340,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":452},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:39.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:39.322 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:40.106 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:40.107 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:40.164 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-11T05:21:39.605797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":205640340,"num_objects":50209,"num_object_clones":0,"num_object_copies":100418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231490,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9663,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160527,"ondisk_log_size":160527,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1642360,"kb_used_data":576876,"kb_used_omap":12,"kb_used_meta":1065395,"kb_avail":281473160,"statfs":{"total":289910292480,"available":288228515840,"internally_reserved":0,"allocated":590721024,"data_stored":589849686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1090964481},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,4,10,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18759680,"num_objects":-4580,"num_object_clones":0,"num_object_copies":-9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4580,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012952"},"pg_stats":[{"pgid":"2.f","version":"47'17013","reported_seq":17097,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.580649+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:37.580649+0000","last_peered":"2024-09-11T05:21:37.580649+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:37.580649+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13012992,"num_objects":3177,"num_object_clones":0,"num_object_copies":6354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17013,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-498}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16814","reported_seq":16922,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.438021+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:37.438021+0000","last_peered":"2024-09-11T05:21:37.438021+0000","last_clean":"2024-09-11T05:21:37.438021+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:37.438021+0000","last_undegraded":"2024-09-11T05:21:37.438021+0000","last_fullsized":"2024-09-11T05:21:37.438021+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12640256,"num_objects":3086,"num_object_clones":0,"num_object_copies":6172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15212,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16604","reported_seq":21893,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.437259+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:37.437259+0000","last_peered":"2024-09-11T05:21:37.437259+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:37.437259+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12750872,"num_objects":3114,"num_object_clones":0,"num_object_copies":6228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15007,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2579,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2198}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16864","reported_seq":16941,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.332454+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:37.332454+0000","last_peered":"2024-09-11T05:21:37.332454+0000","last_clean":"2024-09-11T05:21:37.332454+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:37.332454+0000","last_undegraded":"2024-09-11T05:21:37.332454+0000","last_fullsized":"2024-09-11T05:21:37.332454+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12681216,"num_objects":3096,"num_object_clones":0,"num_object_copies":6192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16864,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17015","reported_seq":17150,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.572016+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:37.572016+0000","last_peered":"2024-09-11T05:21:37.572016+0000","last_clean":"2024-09-11T05:21:37.572016+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:37.572016+0000","last_undegraded":"2024-09-11T05:21:37.572016+0000","last_fullsized":"2024-09-11T05:21:37.572016+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12918784,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13849,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16819","reported_seq":24168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:35.396738+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:35.396738+0000","last_peered":"2024-09-11T05:21:35.396738+0000","last_clean":"2024-09-11T05:21:35.396738+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:35.396738+0000","last_undegraded":"2024-09-11T05:21:35.396738+0000","last_fullsized":"2024-09-11T05:21:35.396738+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13172736,"num_objects":3216,"num_object_clones":0,"num_object_copies":6432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13363,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17024","reported_seq":17162,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.628577+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:37.628577+0000","last_peered":"2024-09-11T05:21:37.628577+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:37.628577+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13017088,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14189,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-398}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16718","reported_seq":16848,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.588993+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:37.588993+0000","last_peered":"2024-09-11T05:21:37.588993+0000","last_clean":"2024-09-11T05:21:37.588993+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:37.588993+0000","last_undegraded":"2024-09-11T05:21:37.588993+0000","last_fullsized":"2024-09-11T05:21:37.588993+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12967936,"num_objects":3166,"num_object_clones":0,"num_object_copies":6332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14021,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16836","reported_seq":16973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.344305+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:37.344305+0000","last_peered":"2024-09-11T05:21:37.344305+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:37.344305+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12840960,"num_objects":3135,"num_object_clones":0,"num_object_copies":6270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-390}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16546","reported_seq":16717,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.535007+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:37.535007+0000","last_peered":"2024-09-11T05:21:37.535007+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:37.535007+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12804096,"num_objects":3126,"num_object_clones":0,"num_object_copies":6252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13299,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-316}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16902","reported_seq":17026,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.420547+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:37.420547+0000","last_peered":"2024-09-11T05:21:37.420547+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:37.420547+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12824576,"num_objects":3131,"num_object_clones":0,"num_object_copies":6262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14931,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-391}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16543","reported_seq":23632,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:37.462575+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:37.462575+0000","last_peered":"2024-09-11T05:21:37.462575+0000","last_clean":"2024-09-11T05:21:37.462575+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:37.462575+0000","last_undegraded":"2024-09-11T05:21:37.462575+0000","last_fullsized":"2024-09-11T05:21:37.462575+0000","mapping_epoch":44,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12963840,"num_objects":3165,"num_object_clones":0,"num_object_copies":6330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14446,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16992","reported_seq":17129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.575638+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:37.575638+0000","last_peered":"2024-09-11T05:21:37.575638+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:37.575638+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12910592,"num_objects":3152,"num_object_clones":0,"num_object_copies":6304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14025,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16566","reported_seq":16705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.634704+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:37.634704+0000","last_peered":"2024-09-11T05:21:37.634704+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:37.634704+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12648448,"num_objects":3088,"num_object_clones":0,"num_object_copies":6176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13432,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16638","reported_seq":16779,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.558039+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:37.558039+0000","last_peered":"2024-09-11T05:21:37.558039+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:37.558039+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12951552,"num_objects":3162,"num_object_clones":0,"num_object_copies":6324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13396,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16530","reported_seq":16680,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:37.619180+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:37.619180+0000","last_peered":"2024-09-11T05:21:37.619180+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:37.619180+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534377,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13537,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":205640321,"num_objects":50207,"num_object_clones":0,"num_object_copies":100414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231488,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9662,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":587776000,"data_stored":587698276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160524,"ondisk_log_size":160524,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513256,"kb_used_data":214980,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93858584,"statfs":{"total":96636764160,"available":96111190016,"internally_reserved":0,"allocated":220139520,"data_stored":219870149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305393323},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515400,"kb_used_data":154680,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93856440,"statfs":{"total":96636764160,"available":96108994560,"internally_reserved":0,"allocated":158392320,"data_stored":158059729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215213,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613704,"kb_used_data":207216,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93758136,"statfs":{"total":96636764160,"available":96008331264,"internally_reserved":0,"allocated":212189184,"data_stored":211919808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,3,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211308544,"data_stored":211305065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157270016,"data_stored":157203322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":219197440,"data_stored":219189889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:43.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:43.955 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:43.955 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:44.016 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-11T05:21:43.607161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":196354708,"num_objects":47942,"num_object_clones":0,"num_object_copies":95884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233757,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9774,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160994,"ondisk_log_size":160994,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1628408,"kb_used_data":558780,"kb_used_omap":12,"kb_used_meta":1069555,"kb_avail":281487112,"statfs":{"total":289910292480,"available":288242802688,"internally_reserved":0,"allocated":572190720,"data_stored":571319382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1095224321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,6,6,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27516928,"num_objects":-6718,"num_object_clones":0,"num_object_copies":-13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6718,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010937"},"pg_stats":[{"pgid":"2.f","version":"47'17166","reported_seq":17250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.617647+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:42.617647+0000","last_peered":"2024-09-11T05:21:42.617647+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:42.617647+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12386304,"num_objects":3024,"num_object_clones":0,"num_object_copies":6048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17166,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16963","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.386807+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:42.386807+0000","last_peered":"2024-09-11T05:21:42.386807+0000","last_clean":"2024-09-11T05:21:42.386807+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:42.386807+0000","last_undegraded":"2024-09-11T05:21:42.386807+0000","last_fullsized":"2024-09-11T05:21:42.386807+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12029952,"num_objects":2937,"num_object_clones":0,"num_object_copies":5874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15361,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16745","reported_seq":22258,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.458899+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:42.458899+0000","last_peered":"2024-09-11T05:21:42.458899+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:42.458899+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12173336,"num_objects":2973,"num_object_clones":0,"num_object_copies":5946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2973,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15148,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2690,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2168}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16991","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.489685+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:42.489685+0000","last_peered":"2024-09-11T05:21:42.489685+0000","last_clean":"2024-09-11T05:21:42.489685+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:42.489685+0000","last_undegraded":"2024-09-11T05:21:42.489685+0000","last_fullsized":"2024-09-11T05:21:42.489685+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12161024,"num_objects":2969,"num_object_clones":0,"num_object_copies":5938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16991,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17156","reported_seq":17292,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.650796+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:42.650796+0000","last_peered":"2024-09-11T05:21:42.650796+0000","last_clean":"2024-09-11T05:21:42.650796+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:42.650796+0000","last_undegraded":"2024-09-11T05:21:42.650796+0000","last_fullsized":"2024-09-11T05:21:42.650796+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12341248,"num_objects":3013,"num_object_clones":0,"num_object_copies":6026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13990,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16982","reported_seq":24333,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:40.464714+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:40.464714+0000","last_peered":"2024-09-11T05:21:40.464714+0000","last_clean":"2024-09-11T05:21:40.464714+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:40.464714+0000","last_undegraded":"2024-09-11T05:21:40.464714+0000","last_fullsized":"2024-09-11T05:21:40.464714+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12505088,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17180","reported_seq":17318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.649261+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:42.649261+0000","last_peered":"2024-09-11T05:21:42.649261+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:42.649261+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12378112,"num_objects":3022,"num_object_clones":0,"num_object_copies":6044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14345,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16841","reported_seq":16971,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.669473+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:42.669473+0000","last_peered":"2024-09-11T05:21:42.669473+0000","last_clean":"2024-09-11T05:21:42.669473+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:42.669473+0000","last_undegraded":"2024-09-11T05:21:42.669473+0000","last_fullsized":"2024-09-11T05:21:42.669473+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12464128,"num_objects":3043,"num_object_clones":0,"num_object_copies":6086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14144,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16984","reported_seq":17121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.474183+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:42.474183+0000","last_peered":"2024-09-11T05:21:42.474183+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:42.474183+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12234752,"num_objects":2987,"num_object_clones":0,"num_object_copies":5974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15052,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16670","reported_seq":16841,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.635559+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:42.635559+0000","last_peered":"2024-09-11T05:21:42.635559+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:42.635559+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12296192,"num_objects":3002,"num_object_clones":0,"num_object_copies":6004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13423,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17017","reported_seq":17140,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.467770+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:42.467770+0000","last_peered":"2024-09-11T05:21:42.467770+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:42.467770+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12353536,"num_objects":3016,"num_object_clones":0,"num_object_copies":6032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15046,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-506}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16680","reported_seq":23771,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.410724+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:42.410724+0000","last_peered":"2024-09-11T05:21:42.410724+0000","last_clean":"2024-09-11T05:21:42.410724+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:42.410724+0000","last_undegraded":"2024-09-11T05:21:42.410724+0000","last_fullsized":"2024-09-11T05:21:42.410724+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12402688,"num_objects":3028,"num_object_clones":0,"num_object_copies":6056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14583,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17147","reported_seq":17284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.570877+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:42.570877+0000","last_peered":"2024-09-11T05:21:42.570877+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:42.570877+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12275712,"num_objects":2997,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14180,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16703","reported_seq":16843,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.690783+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:42.690783+0000","last_peered":"2024-09-11T05:21:42.690783+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:42.690783+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12087296,"num_objects":2951,"num_object_clones":0,"num_object_copies":5902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13569,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16793","reported_seq":16934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.542404+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:42.542404+0000","last_peered":"2024-09-11T05:21:42.542404+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:42.542404+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12316672,"num_objects":3007,"num_object_clones":0,"num_object_copies":6014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13551,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16673","reported_seq":16822,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.689311+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:42.689311+0000","last_peered":"2024-09-11T05:21:42.689311+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:42.689311+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11948649,"num_objects":2918,"num_object_clones":0,"num_object_copies":5836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13680,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":196354689,"num_objects":47940,"num_object_clones":0,"num_object_copies":95880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233755,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9773,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569118720,"data_stored":569040996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160991,"ondisk_log_size":160991,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513140,"kb_used_data":210724,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93858700,"statfs":{"total":96636764160,"available":96111308800,"internally_reserved":0,"allocated":215781376,"data_stored":215512005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500200,"kb_used_data":147788,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93871640,"statfs":{"total":96636764160,"available":96124559360,"internally_reserved":0,"allocated":151334912,"data_stored":151002321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,1,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615068,"kb_used_data":200268,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93756772,"statfs":{"total":96636764160,"available":96006934528,"internally_reserved":0,"allocated":205074432,"data_stored":204805056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204206080,"data_stored":204202601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":150102016,"data_stored":150035322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":214810624,"data_stored":214803073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:44.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:44.904 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:44.961 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-11T05:21:43.607161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":196354708,"num_objects":47942,"num_object_clones":0,"num_object_copies":95884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233757,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9774,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160994,"ondisk_log_size":160994,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1628408,"kb_used_data":558780,"kb_used_omap":12,"kb_used_meta":1069555,"kb_avail":281487112,"statfs":{"total":289910292480,"available":288242802688,"internally_reserved":0,"allocated":572190720,"data_stored":571319382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1095224321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,6,6,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27516928,"num_objects":-6718,"num_object_clones":0,"num_object_copies":-13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6718,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010937"},"pg_stats":[{"pgid":"2.f","version":"47'17166","reported_seq":17250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.617647+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:42.617647+0000","last_peered":"2024-09-11T05:21:42.617647+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:42.617647+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12386304,"num_objects":3024,"num_object_clones":0,"num_object_copies":6048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17166,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16963","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.386807+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:42.386807+0000","last_peered":"2024-09-11T05:21:42.386807+0000","last_clean":"2024-09-11T05:21:42.386807+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:42.386807+0000","last_undegraded":"2024-09-11T05:21:42.386807+0000","last_fullsized":"2024-09-11T05:21:42.386807+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12029952,"num_objects":2937,"num_object_clones":0,"num_object_copies":5874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15361,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16745","reported_seq":22258,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.458899+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:42.458899+0000","last_peered":"2024-09-11T05:21:42.458899+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:42.458899+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12173336,"num_objects":2973,"num_object_clones":0,"num_object_copies":5946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2973,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15148,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2690,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2168}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16991","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.489685+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:42.489685+0000","last_peered":"2024-09-11T05:21:42.489685+0000","last_clean":"2024-09-11T05:21:42.489685+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:42.489685+0000","last_undegraded":"2024-09-11T05:21:42.489685+0000","last_fullsized":"2024-09-11T05:21:42.489685+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12161024,"num_objects":2969,"num_object_clones":0,"num_object_copies":5938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16991,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17156","reported_seq":17292,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.650796+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:42.650796+0000","last_peered":"2024-09-11T05:21:42.650796+0000","last_clean":"2024-09-11T05:21:42.650796+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:42.650796+0000","last_undegraded":"2024-09-11T05:21:42.650796+0000","last_fullsized":"2024-09-11T05:21:42.650796+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12341248,"num_objects":3013,"num_object_clones":0,"num_object_copies":6026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13990,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16982","reported_seq":24333,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:40.464714+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:40.464714+0000","last_peered":"2024-09-11T05:21:40.464714+0000","last_clean":"2024-09-11T05:21:40.464714+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:40.464714+0000","last_undegraded":"2024-09-11T05:21:40.464714+0000","last_fullsized":"2024-09-11T05:21:40.464714+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12505088,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17180","reported_seq":17318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.649261+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:42.649261+0000","last_peered":"2024-09-11T05:21:42.649261+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:42.649261+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12378112,"num_objects":3022,"num_object_clones":0,"num_object_copies":6044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14345,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16841","reported_seq":16971,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.669473+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:42.669473+0000","last_peered":"2024-09-11T05:21:42.669473+0000","last_clean":"2024-09-11T05:21:42.669473+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:42.669473+0000","last_undegraded":"2024-09-11T05:21:42.669473+0000","last_fullsized":"2024-09-11T05:21:42.669473+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12464128,"num_objects":3043,"num_object_clones":0,"num_object_copies":6086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14144,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16984","reported_seq":17121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.474183+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:42.474183+0000","last_peered":"2024-09-11T05:21:42.474183+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:42.474183+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12234752,"num_objects":2987,"num_object_clones":0,"num_object_copies":5974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15052,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16670","reported_seq":16841,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.635559+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:42.635559+0000","last_peered":"2024-09-11T05:21:42.635559+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:42.635559+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12296192,"num_objects":3002,"num_object_clones":0,"num_object_copies":6004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13423,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17017","reported_seq":17140,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.467770+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:42.467770+0000","last_peered":"2024-09-11T05:21:42.467770+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:42.467770+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12353536,"num_objects":3016,"num_object_clones":0,"num_object_copies":6032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15046,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-506}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16680","reported_seq":23771,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.410724+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:42.410724+0000","last_peered":"2024-09-11T05:21:42.410724+0000","last_clean":"2024-09-11T05:21:42.410724+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:42.410724+0000","last_undegraded":"2024-09-11T05:21:42.410724+0000","last_fullsized":"2024-09-11T05:21:42.410724+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12402688,"num_objects":3028,"num_object_clones":0,"num_object_copies":6056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14583,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17147","reported_seq":17284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.570877+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:42.570877+0000","last_peered":"2024-09-11T05:21:42.570877+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:42.570877+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12275712,"num_objects":2997,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14180,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16703","reported_seq":16843,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.690783+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:42.690783+0000","last_peered":"2024-09-11T05:21:42.690783+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:42.690783+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12087296,"num_objects":2951,"num_object_clones":0,"num_object_copies":5902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13569,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16793","reported_seq":16934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.542404+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:42.542404+0000","last_peered":"2024-09-11T05:21:42.542404+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:42.542404+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12316672,"num_objects":3007,"num_object_clones":0,"num_object_copies":6014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13551,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16673","reported_seq":16822,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.689311+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:42.689311+0000","last_peered":"2024-09-11T05:21:42.689311+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:42.689311+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11948649,"num_objects":2918,"num_object_clones":0,"num_object_copies":5836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13680,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":196354689,"num_objects":47940,"num_object_clones":0,"num_object_copies":95880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233755,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9773,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569118720,"data_stored":569040996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160991,"ondisk_log_size":160991,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513140,"kb_used_data":210724,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93858700,"statfs":{"total":96636764160,"available":96111308800,"internally_reserved":0,"allocated":215781376,"data_stored":215512005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500200,"kb_used_data":147788,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93871640,"statfs":{"total":96636764160,"available":96124559360,"internally_reserved":0,"allocated":151334912,"data_stored":151002321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,1,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615068,"kb_used_data":200268,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93756772,"statfs":{"total":96636764160,"available":96006934528,"internally_reserved":0,"allocated":205074432,"data_stored":204805056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204206080,"data_stored":204202601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":150102016,"data_stored":150035322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":214810624,"data_stored":214803073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:44.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:44.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17166", 'reported_seq': 17250, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.617647+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:42.617647+0000', 'last_peered': '2024-09-11T05:21:42.617647+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:42.617647+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12386304, 'num_objects': 3024, 'num_object_clones': 0, 'num_object_copies': 6048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17166, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -651}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:44.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:44.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16745", 'reported_seq': 22258, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.458899+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:42.458899+0000', 'last_peered': '2024-09-11T05:21:42.458899+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:42.458899+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12173336, 'num_objects': 2973, 'num_object_clones': 0, 'num_object_copies': 5946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 805, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2973, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15148, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2690, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2168}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:44.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:44.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'17180", 'reported_seq': 17318, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.649261+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:42.649261+0000', 'last_peered': '2024-09-11T05:21:42.649261+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:42.649261+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12378112, 'num_objects': 3022, 'num_object_clones': 0, 'num_object_copies': 6044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14345, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -554}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:44.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:44.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16984", 'reported_seq': 17121, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.474183+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:42.474183+0000', 'last_peered': '2024-09-11T05:21:42.474183+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:42.474183+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12234752, 'num_objects': 2987, 'num_object_clones': 0, 'num_object_copies': 5974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2987, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15052, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -538}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:44.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:44.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16670", 'reported_seq': 16841, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.635559+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:42.635559+0000', 'last_peered': '2024-09-11T05:21:42.635559+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:42.635559+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12296192, 'num_objects': 3002, 'num_object_clones': 0, 'num_object_copies': 6004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13423, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -440}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:44.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:44.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'17017", 'reported_seq': 17140, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.467770+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:42.467770+0000', 'last_peered': '2024-09-11T05:21:42.467770+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:42.467770+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12353536, 'num_objects': 3016, 'num_object_clones': 0, 'num_object_copies': 6032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15046, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -506}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17147", 'reported_seq': 17284, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.570877+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:42.570877+0000', 'last_peered': '2024-09-11T05:21:42.570877+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:42.570877+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12275712, 'num_objects': 2997, 'num_object_clones': 0, 'num_object_copies': 5994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2997, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14180, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16703", 'reported_seq': 16843, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.690783+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:42.690783+0000', 'last_peered': '2024-09-11T05:21:42.690783+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:42.690783+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12087296, 'num_objects': 2951, 'num_object_clones': 0, 'num_object_copies': 5902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13569, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16793", 'reported_seq': 16934, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.542404+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:42.542404+0000', 'last_peered': '2024-09-11T05:21:42.542404+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:42.542404+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12316672, 'num_objects': 3007, 'num_object_clones': 0, 'num_object_copies': 6014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13551, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -537}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:44.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16673", 'reported_seq': 16822, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:42.689311+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:42.689311+0000', 'last_peered': '2024-09-11T05:21:42.689311+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:42.689311+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11948649, 'num_objects': 2918, 'num_object_clones': 0, 'num_object_copies': 5836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13680, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -597}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:44.966 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:45.753 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:45.754 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32739/95884 objects degraded (34.144%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":583,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":47942,"data_bytes":196354708,"bytes_used":1667489792,"bytes_avail":288242802688,"bytes_total":289910292480,"degraded_objects":32739,"degraded_total":95884,"degraded_ratio":0.34144382795878347,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":343,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":559},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:45.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:45.820 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:46.667 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:46.730 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-11T05:21:45.607655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":196354708,"num_objects":47942,"num_object_clones":0,"num_object_copies":95884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233757,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9774,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160994,"ondisk_log_size":160994,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1628408,"kb_used_data":558780,"kb_used_omap":12,"kb_used_meta":1069555,"kb_avail":281487112,"statfs":{"total":289910292480,"available":288242802688,"internally_reserved":0,"allocated":572190720,"data_stored":571319382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1095224321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,6,6,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18751488,"num_objects":-4578,"num_object_clones":0,"num_object_copies":-9156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4578,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007540"},"pg_stats":[{"pgid":"2.f","version":"47'17166","reported_seq":17250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.617647+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:42.617647+0000","last_peered":"2024-09-11T05:21:42.617647+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:42.617647+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12386304,"num_objects":3024,"num_object_clones":0,"num_object_copies":6048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17166,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16963","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.386807+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:42.386807+0000","last_peered":"2024-09-11T05:21:42.386807+0000","last_clean":"2024-09-11T05:21:42.386807+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:42.386807+0000","last_undegraded":"2024-09-11T05:21:42.386807+0000","last_fullsized":"2024-09-11T05:21:42.386807+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12029952,"num_objects":2937,"num_object_clones":0,"num_object_copies":5874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15361,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16745","reported_seq":22258,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.458899+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:42.458899+0000","last_peered":"2024-09-11T05:21:42.458899+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:42.458899+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12173336,"num_objects":2973,"num_object_clones":0,"num_object_copies":5946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2973,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15148,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2690,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2168}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16991","reported_seq":17068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.489685+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:42.489685+0000","last_peered":"2024-09-11T05:21:42.489685+0000","last_clean":"2024-09-11T05:21:42.489685+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:42.489685+0000","last_undegraded":"2024-09-11T05:21:42.489685+0000","last_fullsized":"2024-09-11T05:21:42.489685+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12161024,"num_objects":2969,"num_object_clones":0,"num_object_copies":5938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16991,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17156","reported_seq":17292,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.650796+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:42.650796+0000","last_peered":"2024-09-11T05:21:42.650796+0000","last_clean":"2024-09-11T05:21:42.650796+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:42.650796+0000","last_undegraded":"2024-09-11T05:21:42.650796+0000","last_fullsized":"2024-09-11T05:21:42.650796+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12341248,"num_objects":3013,"num_object_clones":0,"num_object_copies":6026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13990,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16982","reported_seq":24333,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:40.464714+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:40.464714+0000","last_peered":"2024-09-11T05:21:40.464714+0000","last_clean":"2024-09-11T05:21:40.464714+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:40.464714+0000","last_undegraded":"2024-09-11T05:21:40.464714+0000","last_fullsized":"2024-09-11T05:21:40.464714+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12505088,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17180","reported_seq":17318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.649261+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:42.649261+0000","last_peered":"2024-09-11T05:21:42.649261+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:42.649261+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12378112,"num_objects":3022,"num_object_clones":0,"num_object_copies":6044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14345,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16841","reported_seq":16971,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.669473+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:42.669473+0000","last_peered":"2024-09-11T05:21:42.669473+0000","last_clean":"2024-09-11T05:21:42.669473+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:42.669473+0000","last_undegraded":"2024-09-11T05:21:42.669473+0000","last_fullsized":"2024-09-11T05:21:42.669473+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12464128,"num_objects":3043,"num_object_clones":0,"num_object_copies":6086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14144,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16984","reported_seq":17121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.474183+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:42.474183+0000","last_peered":"2024-09-11T05:21:42.474183+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:42.474183+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12234752,"num_objects":2987,"num_object_clones":0,"num_object_copies":5974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15052,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16670","reported_seq":16841,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.635559+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:42.635559+0000","last_peered":"2024-09-11T05:21:42.635559+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:42.635559+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12296192,"num_objects":3002,"num_object_clones":0,"num_object_copies":6004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13423,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17017","reported_seq":17140,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.467770+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:42.467770+0000","last_peered":"2024-09-11T05:21:42.467770+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:42.467770+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12353536,"num_objects":3016,"num_object_clones":0,"num_object_copies":6032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15046,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-506}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16680","reported_seq":23771,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:42.410724+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:42.410724+0000","last_peered":"2024-09-11T05:21:42.410724+0000","last_clean":"2024-09-11T05:21:42.410724+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:42.410724+0000","last_undegraded":"2024-09-11T05:21:42.410724+0000","last_fullsized":"2024-09-11T05:21:42.410724+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12402688,"num_objects":3028,"num_object_clones":0,"num_object_copies":6056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14583,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17147","reported_seq":17284,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.570877+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:42.570877+0000","last_peered":"2024-09-11T05:21:42.570877+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:42.570877+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12275712,"num_objects":2997,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14180,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16703","reported_seq":16843,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.690783+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:42.690783+0000","last_peered":"2024-09-11T05:21:42.690783+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:42.690783+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12087296,"num_objects":2951,"num_object_clones":0,"num_object_copies":5902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13569,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16793","reported_seq":16934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.542404+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:42.542404+0000","last_peered":"2024-09-11T05:21:42.542404+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:42.542404+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12316672,"num_objects":3007,"num_object_clones":0,"num_object_copies":6014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13551,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16673","reported_seq":16822,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:42.689311+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:42.689311+0000","last_peered":"2024-09-11T05:21:42.689311+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:42.689311+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11948649,"num_objects":2918,"num_object_clones":0,"num_object_copies":5836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13680,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":196354689,"num_objects":47940,"num_object_clones":0,"num_object_copies":95880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":233755,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9773,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569118720,"data_stored":569040996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160991,"ondisk_log_size":160991,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513140,"kb_used_data":210724,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93858700,"statfs":{"total":96636764160,"available":96111308800,"internally_reserved":0,"allocated":215781376,"data_stored":215512005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309653163},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500200,"kb_used_data":147788,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93871640,"statfs":{"total":96636764160,"available":96124559360,"internally_reserved":0,"allocated":151334912,"data_stored":151002321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360836779},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,1,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615068,"kb_used_data":200268,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93756772,"statfs":{"total":96636764160,"available":96006934528,"internally_reserved":0,"allocated":205074432,"data_stored":204805056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204206080,"data_stored":204202601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":150102016,"data_stored":150035322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":214810624,"data_stored":214803073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:49.733 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:50.367 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:50.419 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-11T05:21:49.608547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":187187860,"num_objects":45704,"num_object_clones":0,"num_object_copies":91408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235995,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9874,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645136,"kb_used_data":540828,"kb_used_omap":12,"kb_used_meta":1104243,"kb_avail":281470384,"statfs":{"total":289910292480,"available":288225673216,"internally_reserved":0,"allocated":553807872,"data_stored":552936534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1130744833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,4,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23662592,"num_objects":-5777,"num_object_clones":0,"num_object_copies":-11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5777,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007483"},"pg_stats":[{"pgid":"2.f","version":"47'17307","reported_seq":17390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.696376+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:47.696376+0000","last_peered":"2024-09-11T05:21:47.696376+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:47.696376+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17307,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-792}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17116","reported_seq":17224,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.382153+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:47.382153+0000","last_peered":"2024-09-11T05:21:47.382153+0000","last_clean":"2024-09-11T05:21:47.382153+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:47.382153+0000","last_undegraded":"2024-09-11T05:21:47.382153+0000","last_fullsized":"2024-09-11T05:21:47.382153+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11403264,"num_objects":2784,"num_object_clones":0,"num_object_copies":5568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15514,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16891","reported_seq":22603,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.498606+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:47.498606+0000","last_peered":"2024-09-11T05:21:47.498606+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:47.498606+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575320,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15294,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2790,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17134","reported_seq":17210,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.484351+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:47.484351+0000","last_peered":"2024-09-11T05:21:47.484351+0000","last_clean":"2024-09-11T05:21:47.484351+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:47.484351+0000","last_undegraded":"2024-09-11T05:21:47.484351+0000","last_fullsized":"2024-09-11T05:21:47.484351+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575296,"num_objects":2826,"num_object_clones":0,"num_object_copies":5652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17134,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17293","reported_seq":17429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.702606+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:47.702606+0000","last_peered":"2024-09-11T05:21:47.702606+0000","last_clean":"2024-09-11T05:21:47.702606+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:47.702606+0000","last_undegraded":"2024-09-11T05:21:47.702606+0000","last_fullsized":"2024-09-11T05:21:47.702606+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11780096,"num_objects":2876,"num_object_clones":0,"num_object_copies":5752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14127,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17139","reported_seq":24489,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:45.580668+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:45.580668+0000","last_peered":"2024-09-11T05:21:45.580668+0000","last_clean":"2024-09-11T05:21:45.580668+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:45.580668+0000","last_undegraded":"2024-09-11T05:21:45.580668+0000","last_fullsized":"2024-09-11T05:21:45.580668+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11862016,"num_objects":2896,"num_object_clones":0,"num_object_copies":5792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13683,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17308","reported_seq":17447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.740553+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:47.740553+0000","last_peered":"2024-09-11T05:21:47.740553+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:47.740553+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"44'7300","ondisk_log_start":"44'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11853824,"num_objects":2894,"num_object_clones":0,"num_object_copies":5788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14473,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16966","reported_seq":17096,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.654000+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:47.654000+0000","last_peered":"2024-09-11T05:21:47.654000+0000","last_clean":"2024-09-11T05:21:47.654000+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:47.654000+0000","last_undegraded":"2024-09-11T05:21:47.654000+0000","last_fullsized":"2024-09-11T05:21:47.654000+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11952128,"num_objects":2918,"num_object_clones":0,"num_object_copies":5836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14269,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17138","reported_seq":17276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.468353+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:47.468353+0000","last_peered":"2024-09-11T05:21:47.468353+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:47.468353+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11603968,"num_objects":2833,"num_object_clones":0,"num_object_copies":5666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16788","reported_seq":16959,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.702703+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:47.702703+0000","last_peered":"2024-09-11T05:21:47.702703+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:47.702703+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11812864,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13541,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-558}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17151","reported_seq":17275,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.413586+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:47.413586+0000","last_peered":"2024-09-11T05:21:47.413586+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:47.413586+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-640}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16841","reported_seq":23932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.465745+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:47.465745+0000","last_peered":"2024-09-11T05:21:47.465745+0000","last_clean":"2024-09-11T05:21:47.465745+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:47.465745+0000","last_undegraded":"2024-09-11T05:21:47.465745+0000","last_fullsized":"2024-09-11T05:21:47.465745+0000","mapping_epoch":44,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11743232,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14744,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17299","reported_seq":17436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.733403+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:47.733403+0000","last_peered":"2024-09-11T05:21:47.733403+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:47.733403+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11653120,"num_objects":2845,"num_object_clones":0,"num_object_copies":5690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14332,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16817","reported_seq":16957,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.692548+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:47.692548+0000","last_peered":"2024-09-11T05:21:47.692548+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:47.692548+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620352,"num_objects":2837,"num_object_clones":0,"num_object_copies":5674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13683,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16933","reported_seq":17074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.636379+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:47.636379+0000","last_peered":"2024-09-11T05:21:47.636379+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:47.636379+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11743232,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13691,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16808","reported_seq":16958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.629424+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:47.629424+0000","last_peered":"2024-09-11T05:21:47.629424+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:47.629424+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11395689,"num_objects":2783,"num_object_clones":0,"num_object_copies":5566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":187187841,"num_objects":45702,"num_object_clones":0,"num_object_copies":91404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235993,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9873,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":550612992,"data_stored":550535268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530792,"kb_used_data":206164,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93841048,"statfs":{"total":96636764160,"available":96093233152,"internally_reserved":0,"allocated":211111936,"data_stored":210842565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501700,"kb_used_data":140980,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93870140,"statfs":{"total":96636764160,"available":96123023360,"internally_reserved":0,"allocated":144363520,"data_stored":144030929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215215,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612644,"kb_used_data":193684,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93759196,"statfs":{"total":96636764160,"available":96009416704,"internally_reserved":0,"allocated":198332416,"data_stored":198063040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197509120,"data_stored":197505641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143015936,"data_stored":142949242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":210087936,"data_stored":210080385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:50.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:51.330 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:51.332 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:51.366 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-11T05:21:49.608547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":187187860,"num_objects":45704,"num_object_clones":0,"num_object_copies":91408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235995,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9874,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645136,"kb_used_data":540828,"kb_used_omap":12,"kb_used_meta":1104243,"kb_avail":281470384,"statfs":{"total":289910292480,"available":288225673216,"internally_reserved":0,"allocated":553807872,"data_stored":552936534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1130744833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,4,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23662592,"num_objects":-5777,"num_object_clones":0,"num_object_copies":-11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5777,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007483"},"pg_stats":[{"pgid":"2.f","version":"47'17307","reported_seq":17390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.696376+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:47.696376+0000","last_peered":"2024-09-11T05:21:47.696376+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:47.696376+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17307,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-792}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17116","reported_seq":17224,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.382153+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:47.382153+0000","last_peered":"2024-09-11T05:21:47.382153+0000","last_clean":"2024-09-11T05:21:47.382153+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:47.382153+0000","last_undegraded":"2024-09-11T05:21:47.382153+0000","last_fullsized":"2024-09-11T05:21:47.382153+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11403264,"num_objects":2784,"num_object_clones":0,"num_object_copies":5568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15514,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16891","reported_seq":22603,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.498606+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:47.498606+0000","last_peered":"2024-09-11T05:21:47.498606+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:47.498606+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575320,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15294,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2790,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17134","reported_seq":17210,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.484351+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:47.484351+0000","last_peered":"2024-09-11T05:21:47.484351+0000","last_clean":"2024-09-11T05:21:47.484351+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:47.484351+0000","last_undegraded":"2024-09-11T05:21:47.484351+0000","last_fullsized":"2024-09-11T05:21:47.484351+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575296,"num_objects":2826,"num_object_clones":0,"num_object_copies":5652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17134,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17293","reported_seq":17429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.702606+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:47.702606+0000","last_peered":"2024-09-11T05:21:47.702606+0000","last_clean":"2024-09-11T05:21:47.702606+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:47.702606+0000","last_undegraded":"2024-09-11T05:21:47.702606+0000","last_fullsized":"2024-09-11T05:21:47.702606+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11780096,"num_objects":2876,"num_object_clones":0,"num_object_copies":5752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14127,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17139","reported_seq":24489,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:45.580668+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:45.580668+0000","last_peered":"2024-09-11T05:21:45.580668+0000","last_clean":"2024-09-11T05:21:45.580668+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:45.580668+0000","last_undegraded":"2024-09-11T05:21:45.580668+0000","last_fullsized":"2024-09-11T05:21:45.580668+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11862016,"num_objects":2896,"num_object_clones":0,"num_object_copies":5792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13683,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17308","reported_seq":17447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.740553+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:47.740553+0000","last_peered":"2024-09-11T05:21:47.740553+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:47.740553+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"44'7300","ondisk_log_start":"44'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11853824,"num_objects":2894,"num_object_clones":0,"num_object_copies":5788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14473,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16966","reported_seq":17096,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.654000+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:47.654000+0000","last_peered":"2024-09-11T05:21:47.654000+0000","last_clean":"2024-09-11T05:21:47.654000+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:47.654000+0000","last_undegraded":"2024-09-11T05:21:47.654000+0000","last_fullsized":"2024-09-11T05:21:47.654000+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11952128,"num_objects":2918,"num_object_clones":0,"num_object_copies":5836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14269,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17138","reported_seq":17276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.468353+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:47.468353+0000","last_peered":"2024-09-11T05:21:47.468353+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:47.468353+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11603968,"num_objects":2833,"num_object_clones":0,"num_object_copies":5666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16788","reported_seq":16959,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.702703+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:47.702703+0000","last_peered":"2024-09-11T05:21:47.702703+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:47.702703+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11812864,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13541,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-558}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17151","reported_seq":17275,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.413586+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:47.413586+0000","last_peered":"2024-09-11T05:21:47.413586+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:47.413586+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-640}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16841","reported_seq":23932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.465745+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:47.465745+0000","last_peered":"2024-09-11T05:21:47.465745+0000","last_clean":"2024-09-11T05:21:47.465745+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:47.465745+0000","last_undegraded":"2024-09-11T05:21:47.465745+0000","last_fullsized":"2024-09-11T05:21:47.465745+0000","mapping_epoch":44,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11743232,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14744,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17299","reported_seq":17436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.733403+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:47.733403+0000","last_peered":"2024-09-11T05:21:47.733403+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:47.733403+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11653120,"num_objects":2845,"num_object_clones":0,"num_object_copies":5690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14332,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16817","reported_seq":16957,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.692548+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:47.692548+0000","last_peered":"2024-09-11T05:21:47.692548+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:47.692548+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620352,"num_objects":2837,"num_object_clones":0,"num_object_copies":5674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13683,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16933","reported_seq":17074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.636379+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:47.636379+0000","last_peered":"2024-09-11T05:21:47.636379+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:47.636379+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11743232,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13691,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16808","reported_seq":16958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.629424+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:47.629424+0000","last_peered":"2024-09-11T05:21:47.629424+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:47.629424+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11395689,"num_objects":2783,"num_object_clones":0,"num_object_copies":5566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":187187841,"num_objects":45702,"num_object_clones":0,"num_object_copies":91404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235993,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9873,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":550612992,"data_stored":550535268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530792,"kb_used_data":206164,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93841048,"statfs":{"total":96636764160,"available":96093233152,"internally_reserved":0,"allocated":211111936,"data_stored":210842565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501700,"kb_used_data":140980,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93870140,"statfs":{"total":96636764160,"available":96123023360,"internally_reserved":0,"allocated":144363520,"data_stored":144030929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369356459},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215215,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612644,"kb_used_data":193684,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93759196,"statfs":{"total":96636764160,"available":96009416704,"internally_reserved":0,"allocated":198332416,"data_stored":198063040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197509120,"data_stored":197505641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":143015936,"data_stored":142949242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":210087936,"data_stored":210080385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:51.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:51.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17307", 'reported_seq': 17390, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.696376+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:47.696376+0000', 'last_peered': '2024-09-11T05:21:47.696376+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:47.696376+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11808768, 'num_objects': 2883, 'num_object_clones': 0, 'num_object_copies': 5766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2883, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17307, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -792}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16891", 'reported_seq': 22603, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.498606+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:47.498606+0000', 'last_peered': '2024-09-11T05:21:47.498606+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:47.498606+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11575320, 'num_objects': 2827, 'num_object_clones': 0, 'num_object_copies': 5654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 705, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2827, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15294, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2790, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'17308", 'reported_seq': 17447, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.740553+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:47.740553+0000', 'last_peered': '2024-09-11T05:21:47.740553+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:47.740553+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "44'7300", 'ondisk_log_start': "44'7300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11853824, 'num_objects': 2894, 'num_object_clones': 0, 'num_object_copies': 5788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14473, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -682}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17138", 'reported_seq': 17276, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.468353+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:47.468353+0000', 'last_peered': '2024-09-11T05:21:47.468353+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:47.468353+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11603968, 'num_objects': 2833, 'num_object_clones': 0, 'num_object_copies': 5666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15206, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -692}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16788", 'reported_seq': 16959, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.702703+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:47.702703+0000', 'last_peered': '2024-09-11T05:21:47.702703+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:47.702703+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11812864, 'num_objects': 2884, 'num_object_clones': 0, 'num_object_copies': 5768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2884, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13541, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -558}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:51.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'17151", 'reported_seq': 17275, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.413586+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:47.413586+0000', 'last_peered': '2024-09-11T05:21:47.413586+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:47.413586+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11804672, 'num_objects': 2882, 'num_object_clones': 0, 'num_object_copies': 5764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15180, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -640}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17299", 'reported_seq': 17436, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.733403+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:47.733403+0000', 'last_peered': '2024-09-11T05:21:47.733403+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:47.733403+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11653120, 'num_objects': 2845, 'num_object_clones': 0, 'num_object_copies': 5690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14332, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16817", 'reported_seq': 16957, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.692548+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:47.692548+0000', 'last_peered': '2024-09-11T05:21:47.692548+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:47.692548+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11620352, 'num_objects': 2837, 'num_object_clones': 0, 'num_object_copies': 5674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13683, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16933", 'reported_seq': 17074, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.636379+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:47.636379+0000', 'last_peered': '2024-09-11T05:21:47.636379+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:47.636379+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11743232, 'num_objects': 2867, 'num_object_clones': 0, 'num_object_copies': 5734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13691, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:51.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16808", 'reported_seq': 16958, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:47.629424+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:47.629424+0000', 'last_peered': '2024-09-11T05:21:47.629424+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:47.629424+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11395689, 'num_objects': 2783, 'num_object_clones': 0, 'num_object_copies': 5566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2783, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13815, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -732}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:51.370 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:52.463 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:52.463 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32639/91118 objects degraded (35.821%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":590,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":45559,"data_bytes":186593940,"bytes_used":1681973248,"bytes_avail":288228319232,"bytes_total":289910292480,"degraded_objects":32639,"degraded_total":91118,"degraded_ratio":0.35820584297284841,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:52.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:52.538 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:53.413 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:53.467 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-11T05:21:51.609010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186593940,"num_objects":45559,"num_object_clones":0,"num_object_copies":91118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236140,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9874,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1642552,"kb_used_data":534084,"kb_used_omap":12,"kb_used_meta":1108403,"kb_avail":281472968,"statfs":{"total":289910292480,"available":288228319232,"internally_reserved":0,"allocated":546902016,"data_stored":546030678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1135004673},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,5,4,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19046400,"num_objects":-4650,"num_object_clones":0,"num_object_copies":-9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4650,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"2.f","version":"47'17307","reported_seq":17390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.696376+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:47.696376+0000","last_peered":"2024-09-11T05:21:47.696376+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:47.696376+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17307,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-792}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17116","reported_seq":17224,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.382153+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:47.382153+0000","last_peered":"2024-09-11T05:21:47.382153+0000","last_clean":"2024-09-11T05:21:47.382153+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:47.382153+0000","last_undegraded":"2024-09-11T05:21:47.382153+0000","last_fullsized":"2024-09-11T05:21:47.382153+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11403264,"num_objects":2784,"num_object_clones":0,"num_object_copies":5568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15514,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16891","reported_seq":22603,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.498606+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:47.498606+0000","last_peered":"2024-09-11T05:21:47.498606+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:47.498606+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575320,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15294,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2790,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17134","reported_seq":17210,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.484351+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:47.484351+0000","last_peered":"2024-09-11T05:21:47.484351+0000","last_clean":"2024-09-11T05:21:47.484351+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:47.484351+0000","last_undegraded":"2024-09-11T05:21:47.484351+0000","last_fullsized":"2024-09-11T05:21:47.484351+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575296,"num_objects":2826,"num_object_clones":0,"num_object_copies":5652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17134,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17293","reported_seq":17429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.702606+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:47.702606+0000","last_peered":"2024-09-11T05:21:47.702606+0000","last_clean":"2024-09-11T05:21:47.702606+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:47.702606+0000","last_undegraded":"2024-09-11T05:21:47.702606+0000","last_fullsized":"2024-09-11T05:21:47.702606+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11780096,"num_objects":2876,"num_object_clones":0,"num_object_copies":5752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14127,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17284","reported_seq":24636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:50.672298+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:50.672298+0000","last_peered":"2024-09-11T05:21:50.672298+0000","last_clean":"2024-09-11T05:21:50.672298+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:50.672298+0000","last_undegraded":"2024-09-11T05:21:50.672298+0000","last_fullsized":"2024-09-11T05:21:50.672298+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268096,"num_objects":2751,"num_object_clones":0,"num_object_copies":5502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13828,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17308","reported_seq":17447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.740553+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:47.740553+0000","last_peered":"2024-09-11T05:21:47.740553+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:47.740553+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"44'7300","ondisk_log_start":"44'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11853824,"num_objects":2894,"num_object_clones":0,"num_object_copies":5788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14473,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16966","reported_seq":17096,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.654000+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:47.654000+0000","last_peered":"2024-09-11T05:21:47.654000+0000","last_clean":"2024-09-11T05:21:47.654000+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:47.654000+0000","last_undegraded":"2024-09-11T05:21:47.654000+0000","last_fullsized":"2024-09-11T05:21:47.654000+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11952128,"num_objects":2918,"num_object_clones":0,"num_object_copies":5836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14269,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17138","reported_seq":17276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.468353+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:47.468353+0000","last_peered":"2024-09-11T05:21:47.468353+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:47.468353+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11603968,"num_objects":2833,"num_object_clones":0,"num_object_copies":5666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15206,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16788","reported_seq":16959,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.702703+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:47.702703+0000","last_peered":"2024-09-11T05:21:47.702703+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:47.702703+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11812864,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13541,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-558}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17151","reported_seq":17275,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.413586+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:47.413586+0000","last_peered":"2024-09-11T05:21:47.413586+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:47.413586+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-640}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16841","reported_seq":23932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:47.465745+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:47.465745+0000","last_peered":"2024-09-11T05:21:47.465745+0000","last_clean":"2024-09-11T05:21:47.465745+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:47.465745+0000","last_undegraded":"2024-09-11T05:21:47.465745+0000","last_fullsized":"2024-09-11T05:21:47.465745+0000","mapping_epoch":44,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11743232,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14744,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17299","reported_seq":17436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.733403+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:47.733403+0000","last_peered":"2024-09-11T05:21:47.733403+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:47.733403+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11653120,"num_objects":2845,"num_object_clones":0,"num_object_copies":5690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14332,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16817","reported_seq":16957,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.692548+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:47.692548+0000","last_peered":"2024-09-11T05:21:47.692548+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:47.692548+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620352,"num_objects":2837,"num_object_clones":0,"num_object_copies":5674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13683,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16933","reported_seq":17074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.636379+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:47.636379+0000","last_peered":"2024-09-11T05:21:47.636379+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:47.636379+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11743232,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13691,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-677}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16808","reported_seq":16958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:47.629424+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:47.629424+0000","last_peered":"2024-09-11T05:21:47.629424+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:47.629424+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11395689,"num_objects":2783,"num_object_clones":0,"num_object_copies":5566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":186593921,"num_objects":45557,"num_object_clones":0,"num_object_copies":91114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236138,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9873,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543600640,"data_stored":543522916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530792,"kb_used_data":206164,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93841048,"statfs":{"total":96636764160,"available":96093233152,"internally_reserved":0,"allocated":211111936,"data_stored":210842565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499116,"kb_used_data":134236,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93872724,"statfs":{"total":96636764160,"available":96125669376,"internally_reserved":0,"allocated":137457664,"data_stored":137125073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215215,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612644,"kb_used_data":193684,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93759196,"statfs":{"total":96636764160,"available":96009416704,"internally_reserved":0,"allocated":198332416,"data_stored":198063040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197509120,"data_stored":197505641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136003584,"data_stored":135936890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":210087936,"data_stored":210080385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:56.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:57.241 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:57.241 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:57.310 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-11T05:21:55.609971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":177685140,"num_objects":43384,"num_object_clones":0,"num_object_copies":86768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238315,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9989,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1626836,"kb_used_data":522528,"kb_used_omap":12,"kb_used_meta":1104243,"kb_avail":281488684,"statfs":{"total":289910292480,"available":288244412416,"internally_reserved":0,"allocated":535068672,"data_stored":534197334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1130744833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,10,6,8,5,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18669568,"num_objects":-4558,"num_object_clones":0,"num_object_copies":-9116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4558,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002809"},"pg_stats":[{"pgid":"2.f","version":"47'17453","reported_seq":17537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.735494+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:52.735494+0000","last_peered":"2024-09-11T05:21:52.735494+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:52.735494+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11210752,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17453,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17258","reported_seq":17366,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:52.537926+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:52.537926+0000","last_peered":"2024-09-11T05:21:52.537926+0000","last_clean":"2024-09-11T05:21:52.537926+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:52.537926+0000","last_undegraded":"2024-09-11T05:21:52.537926+0000","last_fullsized":"2024-09-11T05:21:52.537926+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10821632,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15656,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17034","reported_seq":22977,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.524377+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:52.524377+0000","last_peered":"2024-09-11T05:21:52.524377+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:52.524377+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10989592,"num_objects":2684,"num_object_clones":0,"num_object_copies":5368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15437,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2905,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2094}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17289","reported_seq":17364,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:52.548568+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:52.548568+0000","last_peered":"2024-09-11T05:21:52.548568+0000","last_clean":"2024-09-11T05:21:52.548568+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:52.548568+0000","last_undegraded":"2024-09-11T05:21:52.548568+0000","last_fullsized":"2024-09-11T05:21:52.548568+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10940416,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17289,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17438","reported_seq":17574,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:52.601863+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:52.601863+0000","last_peered":"2024-09-11T05:21:52.601863+0000","last_clean":"2024-09-11T05:21:52.601863+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:52.601863+0000","last_undegraded":"2024-09-11T05:21:52.601863+0000","last_fullsized":"2024-09-11T05:21:52.601863+0000","mapping_epoch":34,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11186176,"num_objects":2731,"num_object_clones":0,"num_object_copies":5462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17284","reported_seq":24636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:50.672298+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:50.672298+0000","last_peered":"2024-09-11T05:21:50.672298+0000","last_clean":"2024-09-11T05:21:50.672298+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:50.672298+0000","last_undegraded":"2024-09-11T05:21:50.672298+0000","last_fullsized":"2024-09-11T05:21:50.672298+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268096,"num_objects":2751,"num_object_clones":0,"num_object_copies":5502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13828,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17449","reported_seq":17583,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.638250+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:52.638250+0000","last_peered":"2024-09-11T05:21:52.638250+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:52.638250+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11276288,"num_objects":2753,"num_object_clones":0,"num_object_copies":5506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14614,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17129","reported_seq":17259,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:52.731408+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:52.731408+0000","last_peered":"2024-09-11T05:21:52.731408+0000","last_clean":"2024-09-11T05:21:52.731408+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:52.731408+0000","last_undegraded":"2024-09-11T05:21:52.731408+0000","last_fullsized":"2024-09-11T05:21:52.731408+0000","mapping_epoch":34,"log_start":"44'7100","ondisk_log_start":"44'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11284480,"num_objects":2755,"num_object_clones":0,"num_object_copies":5510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14432,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17277","reported_seq":17415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.447447+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:52.447447+0000","last_peered":"2024-09-11T05:21:52.447447+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:52.447447+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11034624,"num_objects":2694,"num_object_clones":0,"num_object_copies":5388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15345,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-831}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16940","reported_seq":17111,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.712610+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:52.712610+0000","last_peered":"2024-09-11T05:21:52.712610+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:52.712610+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11190272,"num_objects":2732,"num_object_clones":0,"num_object_copies":5464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17303","reported_seq":17427,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.567482+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:52.567482+0000","last_peered":"2024-09-11T05:21:52.567482+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:52.567482+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11182080,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15332,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-792}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16987","reported_seq":24079,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:52.543827+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:52.543827+0000","last_peered":"2024-09-11T05:21:52.543827+0000","last_clean":"2024-09-11T05:21:52.543827+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:52.543827+0000","last_undegraded":"2024-09-11T05:21:52.543827+0000","last_fullsized":"2024-09-11T05:21:52.543827+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145216,"num_objects":2721,"num_object_clones":0,"num_object_copies":5442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14890,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17434","reported_seq":17572,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.734791+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:52.734791+0000","last_peered":"2024-09-11T05:21:52.734791+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:52.734791+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11100160,"num_objects":2710,"num_object_clones":0,"num_object_copies":5420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14467,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-913}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16944","reported_seq":17082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.734342+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:52.734342+0000","last_peered":"2024-09-11T05:21:52.734342+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:52.734342+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11100160,"num_objects":2710,"num_object_clones":0,"num_object_copies":5420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13810,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17083","reported_seq":17224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.680820+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:52.680820+0000","last_peered":"2024-09-11T05:21:52.680820+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:52.680820+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11128832,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13841,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-827}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16947","reported_seq":17097,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.662764+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:52.662764+0000","last_peered":"2024-09-11T05:21:52.662764+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:52.662764+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10826345,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-871}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":177685121,"num_objects":43382,"num_object_clones":0,"num_object_copies":86764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238313,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9988,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":531656704,"data_stored":531578980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530312,"kb_used_data":201528,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93841528,"statfs":{"total":96636764160,"available":96093724672,"internally_reserved":0,"allocated":206364672,"data_stored":206095301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499116,"kb_used_data":134236,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93872724,"statfs":{"total":96636764160,"available":96125669376,"internally_reserved":0,"allocated":137457664,"data_stored":137125073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373616299},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597408,"kb_used_data":186764,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93774432,"statfs":{"total":96636764160,"available":96025018368,"internally_reserved":0,"allocated":191246336,"data_stored":190976960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,3,0,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":190439424,"data_stored":190435945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136003584,"data_stored":135936890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205213696,"data_stored":205206145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:57.311 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:58.021 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:58.024 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:21:58.074 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-11T05:21:57.616255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173552276,"num_objects":42375,"num_object_clones":0,"num_object_copies":84750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239324,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10108,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1594508,"kb_used_data":510996,"kb_used_omap":12,"kb_used_meta":1083443,"kb_avail":281521012,"statfs":{"total":289910292480,"available":288277516288,"internally_reserved":0,"allocated":523259904,"data_stored":522388566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1109445633},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,10,2,7,5,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22802432,"num_objects":-5567,"num_object_clones":0,"num_object_copies":-11134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5567,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008599"},"pg_stats":[{"pgid":"2.f","version":"47'17453","reported_seq":17537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.735494+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:52.735494+0000","last_peered":"2024-09-11T05:21:52.735494+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:52.735494+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11210752,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17453,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17399","reported_seq":17507,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.413281+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:57.413281+0000","last_peered":"2024-09-11T05:21:57.413281+0000","last_clean":"2024-09-11T05:21:57.413281+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:57.413281+0000","last_undegraded":"2024-09-11T05:21:57.413281+0000","last_fullsized":"2024-09-11T05:21:57.413281+0000","mapping_epoch":35,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10244096,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15797,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17177","reported_seq":23358,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.584502+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:57.584502+0000","last_peered":"2024-09-11T05:21:57.584502+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:57.584502+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"44'7100","ondisk_log_start":"44'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403864,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15580,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3024,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17425","reported_seq":17502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.550053+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:57.550053+0000","last_peered":"2024-09-11T05:21:57.550053+0000","last_clean":"2024-09-11T05:21:57.550053+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:57.550053+0000","last_undegraded":"2024-09-11T05:21:57.550053+0000","last_fullsized":"2024-09-11T05:21:57.550053+0000","mapping_epoch":35,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10383360,"num_objects":2535,"num_object_clones":0,"num_object_copies":5070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17425,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17438","reported_seq":17574,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:52.601863+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:52.601863+0000","last_peered":"2024-09-11T05:21:52.601863+0000","last_clean":"2024-09-11T05:21:52.601863+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:52.601863+0000","last_undegraded":"2024-09-11T05:21:52.601863+0000","last_fullsized":"2024-09-11T05:21:52.601863+0000","mapping_epoch":34,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11186176,"num_objects":2731,"num_object_clones":0,"num_object_copies":5462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17432","reported_seq":24784,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:55.715360+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:55.715360+0000","last_peered":"2024-09-11T05:21:55.715360+0000","last_clean":"2024-09-11T05:21:55.715360+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:55.715360+0000","last_undegraded":"2024-09-11T05:21:55.715360+0000","last_fullsized":"2024-09-11T05:21:55.715360+0000","mapping_epoch":46,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10661888,"num_objects":2603,"num_object_clones":0,"num_object_copies":5206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17449","reported_seq":17583,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.638250+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:52.638250+0000","last_peered":"2024-09-11T05:21:52.638250+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:52.638250+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11276288,"num_objects":2753,"num_object_clones":0,"num_object_copies":5506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14614,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17129","reported_seq":17259,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:52.731408+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:52.731408+0000","last_peered":"2024-09-11T05:21:52.731408+0000","last_clean":"2024-09-11T05:21:52.731408+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:52.731408+0000","last_undegraded":"2024-09-11T05:21:52.731408+0000","last_fullsized":"2024-09-11T05:21:52.731408+0000","mapping_epoch":34,"log_start":"44'7100","ondisk_log_start":"44'7100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11284480,"num_objects":2755,"num_object_clones":0,"num_object_copies":5510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14432,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17413","reported_seq":17551,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.515216+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:57.515216+0000","last_peered":"2024-09-11T05:21:57.515216+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:57.515216+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10477568,"num_objects":2558,"num_object_clones":0,"num_object_copies":5116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15481,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-967}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16940","reported_seq":17111,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.712610+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:52.712610+0000","last_peered":"2024-09-11T05:21:52.712610+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:52.712610+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11190272,"num_objects":2732,"num_object_clones":0,"num_object_copies":5464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17455","reported_seq":17579,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.593400+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:57.593400+0000","last_peered":"2024-09-11T05:21:57.593400+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:57.593400+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10559488,"num_objects":2578,"num_object_clones":0,"num_object_copies":5156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15484,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17140","reported_seq":24231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.524712+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:57.524712+0000","last_peered":"2024-09-11T05:21:57.524712+0000","last_clean":"2024-09-11T05:21:57.524712+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:57.524712+0000","last_undegraded":"2024-09-11T05:21:57.524712+0000","last_fullsized":"2024-09-11T05:21:57.524712+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15043,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17434","reported_seq":17572,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.734791+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:52.734791+0000","last_peered":"2024-09-11T05:21:52.734791+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:52.734791+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11100160,"num_objects":2710,"num_object_clones":0,"num_object_copies":5420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14467,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-913}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16944","reported_seq":17082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.734342+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:52.734342+0000","last_peered":"2024-09-11T05:21:52.734342+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:52.734342+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11100160,"num_objects":2710,"num_object_clones":0,"num_object_copies":5420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13810,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17083","reported_seq":17224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.680820+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:52.680820+0000","last_peered":"2024-09-11T05:21:52.680820+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:52.680820+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11128832,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13841,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-827}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16947","reported_seq":17097,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:52.662764+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:52.662764+0000","last_peered":"2024-09-11T05:21:52.662764+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:52.662764+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10826345,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-871}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173552257,"num_objects":42373,"num_object_clones":0,"num_object_copies":84746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239322,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10107,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":519811072,"data_stored":519733348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513160,"kb_used_data":196856,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93858680,"statfs":{"total":96636764160,"available":96111288320,"internally_reserved":0,"allocated":201580544,"data_stored":201311173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483940,"kb_used_data":127376,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93887900,"statfs":{"total":96636764160,"available":96141209600,"internally_reserved":0,"allocated":130433024,"data_stored":130100433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597408,"kb_used_data":186764,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93774432,"statfs":{"total":96636764160,"available":96025018368,"internally_reserved":0,"allocated":191246336,"data_stored":190976960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,3,0,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":190439424,"data_stored":190435945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":128888832,"data_stored":128822138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200482816,"data_stored":200475265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:21:58.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:21:58.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17453", 'reported_seq': 17537, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:52.735494+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:21:52.735494+0000', 'last_peered': '2024-09-11T05:21:52.735494+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:21:52.735494+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11210752, 'num_objects': 2737, 'num_object_clones': 0, 'num_object_copies': 5474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17453, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -938}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:58.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:21:58.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17177", 'reported_seq': 23358, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:57.584502+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:21:57.584502+0000', 'last_peered': '2024-09-11T05:21:57.584502+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:21:57.584502+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10403864, 'num_objects': 2541, 'num_object_clones': 0, 'num_object_copies': 5082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2541, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15580, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3024, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'17449", 'reported_seq': 17583, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:52.638250+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:21:52.638250+0000', 'last_peered': '2024-09-11T05:21:52.638250+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:21:52.638250+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11276288, 'num_objects': 2753, 'num_object_clones': 0, 'num_object_copies': 5506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14614, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -823}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17413", 'reported_seq': 17551, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:57.515216+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:21:57.515216+0000', 'last_peered': '2024-09-11T05:21:57.515216+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:21:57.515216+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10477568, 'num_objects': 2558, 'num_object_clones': 0, 'num_object_copies': 5116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15481, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -967}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16940", 'reported_seq': 17111, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:52.712610+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:21:52.712610+0000', 'last_peered': '2024-09-11T05:21:52.712610+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:21:52.712610+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11190272, 'num_objects': 2732, 'num_object_clones': 0, 'num_object_copies': 5464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13693, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -710}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'17455", 'reported_seq': 17579, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:57.593400+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:21:57.593400+0000', 'last_peered': '2024-09-11T05:21:57.593400+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:21:57.593400+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10559488, 'num_objects': 2578, 'num_object_clones': 0, 'num_object_copies': 5156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15484, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -944}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:21:58.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17434", 'reported_seq': 17572, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:52.734791+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:21:52.734791+0000', 'last_peered': '2024-09-11T05:21:52.734791+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:21:52.734791+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11100160, 'num_objects': 2710, 'num_object_clones': 0, 'num_object_copies': 5420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14467, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -913}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:58.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:21:58.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16944", 'reported_seq': 17082, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:52.734342+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:21:52.734342+0000', 'last_peered': '2024-09-11T05:21:52.734342+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:21:52.734342+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11100160, 'num_objects': 2710, 'num_object_clones': 0, 'num_object_copies': 5420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13810, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:58.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:21:58.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17083", 'reported_seq': 17224, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:52.680820+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:21:52.680820+0000', 'last_peered': '2024-09-11T05:21:52.680820+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:21:52.680820+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11128832, 'num_objects': 2717, 'num_object_clones': 0, 'num_object_copies': 5434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13841, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -827}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:58.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:21:58.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16947", 'reported_seq': 17097, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:21:52.662764+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:21:52.662764+0000', 'last_peered': '2024-09-11T05:21:52.662764+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:21:52.662764+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10826345, 'num_objects': 2644, 'num_object_clones': 0, 'num_object_copies': 5288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13954, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -871}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:21:58.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:21:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:59.080 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32405/84750 objects degraded (38.236%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":596,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":42375,"data_bytes":173552276,"bytes_used":1632776192,"bytes_avail":288277516288,"bytes_total":289910292480,"degraded_objects":32405,"degraded_total":84750,"degraded_ratio":0.38235988200589971,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":334,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:21:59.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:21:59.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:21:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:21:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:00.035 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-11T05:21:59.616716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":168215188,"num_objects":41072,"num_object_clones":0,"num_object_copies":82144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":240627,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10108,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1591784,"kb_used_data":504112,"kb_used_omap":12,"kb_used_meta":1087603,"kb_avail":281523736,"statfs":{"total":289910292480,"available":288280305664,"internally_reserved":0,"allocated":516210688,"data_stored":515339350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1113705473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,4,7,9,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23846912,"num_objects":-5822,"num_object_clones":0,"num_object_copies":-11644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5822,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008647"},"pg_stats":[{"pgid":"2.f","version":"47'17604","reported_seq":17685,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.777098+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:21:57.777098+0000","last_peered":"2024-09-11T05:21:57.777098+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:21:57.777098+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10592256,"num_objects":2586,"num_object_clones":0,"num_object_copies":5172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17604,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17399","reported_seq":17507,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.413281+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:21:57.413281+0000","last_peered":"2024-09-11T05:21:57.413281+0000","last_clean":"2024-09-11T05:21:57.413281+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:21:57.413281+0000","last_undegraded":"2024-09-11T05:21:57.413281+0000","last_fullsized":"2024-09-11T05:21:57.413281+0000","mapping_epoch":35,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10244096,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15797,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17177","reported_seq":23358,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.584502+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:21:57.584502+0000","last_peered":"2024-09-11T05:21:57.584502+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:21:57.584502+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"44'7100","ondisk_log_start":"44'7100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403864,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15580,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3024,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17425","reported_seq":17502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.550053+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:21:57.550053+0000","last_peered":"2024-09-11T05:21:57.550053+0000","last_clean":"2024-09-11T05:21:57.550053+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:21:57.550053+0000","last_undegraded":"2024-09-11T05:21:57.550053+0000","last_fullsized":"2024-09-11T05:21:57.550053+0000","mapping_epoch":35,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10383360,"num_objects":2535,"num_object_clones":0,"num_object_copies":5070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17425,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17590","reported_seq":17726,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.795449+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:21:57.795449+0000","last_peered":"2024-09-11T05:21:57.795449+0000","last_clean":"2024-09-11T05:21:57.795449+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:21:57.795449+0000","last_undegraded":"2024-09-11T05:21:57.795449+0000","last_fullsized":"2024-09-11T05:21:57.795449+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10563584,"num_objects":2579,"num_object_clones":0,"num_object_copies":5158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14424,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17432","reported_seq":24784,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:55.715360+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:21:55.715360+0000","last_peered":"2024-09-11T05:21:55.715360+0000","last_clean":"2024-09-11T05:21:55.715360+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:21:55.715360+0000","last_undegraded":"2024-09-11T05:21:55.715360+0000","last_fullsized":"2024-09-11T05:21:55.715360+0000","mapping_epoch":46,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10661888,"num_objects":2603,"num_object_clones":0,"num_object_copies":5206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17584","reported_seq":17723,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.608214+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:21:57.608214+0000","last_peered":"2024-09-11T05:21:57.608214+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:21:57.608214+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723328,"num_objects":2618,"num_object_clones":0,"num_object_copies":5236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14749,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-958}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17271","reported_seq":17401,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.754862+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:21:57.754862+0000","last_peered":"2024-09-11T05:21:57.754862+0000","last_clean":"2024-09-11T05:21:57.754862+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:21:57.754862+0000","last_undegraded":"2024-09-11T05:21:57.754862+0000","last_fullsized":"2024-09-11T05:21:57.754862+0000","mapping_epoch":34,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10702848,"num_objects":2613,"num_object_clones":0,"num_object_copies":5226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14574,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17413","reported_seq":17551,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.515216+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:21:57.515216+0000","last_peered":"2024-09-11T05:21:57.515216+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:21:57.515216+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10477568,"num_objects":2558,"num_object_clones":0,"num_object_copies":5116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15481,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-967}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17095","reported_seq":17266,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.797006+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:21:57.797006+0000","last_peered":"2024-09-11T05:21:57.797006+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:21:57.797006+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10555392,"num_objects":2577,"num_object_clones":0,"num_object_copies":5154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13848,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-865}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17455","reported_seq":17579,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.593400+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:21:57.593400+0000","last_peered":"2024-09-11T05:21:57.593400+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:21:57.593400+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10559488,"num_objects":2578,"num_object_clones":0,"num_object_copies":5156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15484,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17140","reported_seq":24231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:21:57.524712+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:21:57.524712+0000","last_peered":"2024-09-11T05:21:57.524712+0000","last_clean":"2024-09-11T05:21:57.524712+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:21:57.524712+0000","last_undegraded":"2024-09-11T05:21:57.524712+0000","last_fullsized":"2024-09-11T05:21:57.524712+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15043,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17572","reported_seq":17710,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.751182+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:21:57.751182+0000","last_peered":"2024-09-11T05:21:57.751182+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:21:57.751182+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10534912,"num_objects":2572,"num_object_clones":0,"num_object_copies":5144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14605,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17089","reported_seq":17229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.765619+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:21:57.765619+0000","last_peered":"2024-09-11T05:21:57.765619+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:21:57.765619+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506240,"num_objects":2565,"num_object_clones":0,"num_object_copies":5130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13955,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-930}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17232","reported_seq":17373,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.671593+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:21:57.671593+0000","last_peered":"2024-09-11T05:21:57.671593+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:21:57.671593+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13990,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17083","reported_seq":17233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:21:57.772470+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:21:57.772470+0000","last_peered":"2024-09-11T05:21:57.772470+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:21:57.772470+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10269289,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14090,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1007}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":168215169,"num_objects":41070,"num_object_clones":0,"num_object_copies":82140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":240625,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10107,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512692224,"data_stored":512614500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513160,"kb_used_data":196856,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93858680,"statfs":{"total":96636764160,"available":96111288320,"internally_reserved":0,"allocated":201580544,"data_stored":201311173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483940,"kb_used_data":127376,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93887900,"statfs":{"total":96636764160,"available":96141209600,"internally_reserved":0,"allocated":130433024,"data_stored":130100433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365096619},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215217,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594684,"kb_used_data":179880,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93777156,"statfs":{"total":96636764160,"available":96027807744,"internally_reserved":0,"allocated":184197120,"data_stored":183927744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,4,3,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":183320576,"data_stored":183317097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":128888832,"data_stored":128822138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200482816,"data_stored":200475265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:03.038 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:03.795 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:03.796 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:03.844 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-11T05:22:03.617634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":158769812,"num_objects":38766,"num_object_clones":0,"num_object_copies":77532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242933,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10242,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1604304,"kb_used_data":486108,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281511216,"statfs":{"total":289910292480,"available":288267485184,"internally_reserved":0,"allocated":497774592,"data_stored":496903254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1144966145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,3,12,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27824128,"num_objects":-6793,"num_object_clones":0,"num_object_copies":-13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6793,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008624"},"pg_stats":[{"pgid":"2.f","version":"47'17754","reported_seq":17837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.826740+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:02.826740+0000","last_peered":"2024-09-11T05:22:02.826740+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:02.826740+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9977856,"num_objects":2436,"num_object_clones":0,"num_object_copies":4872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17754,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17527","reported_seq":17635,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.553339+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:02.553339+0000","last_peered":"2024-09-11T05:22:02.553339+0000","last_clean":"2024-09-11T05:22:02.553339+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:02.553339+0000","last_undegraded":"2024-09-11T05:22:02.553339+0000","last_fullsized":"2024-09-11T05:22:02.553339+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15925,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17333","reported_seq":23782,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.554358+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:02.554358+0000","last_peered":"2024-09-11T05:22:02.554358+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:02.554358+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764888,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15736,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3158,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2048}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17577","reported_seq":17654,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.529115+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:02.529115+0000","last_peered":"2024-09-11T05:22:02.529115+0000","last_clean":"2024-09-11T05:22:02.529115+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:02.529115+0000","last_undegraded":"2024-09-11T05:22:02.529115+0000","last_fullsized":"2024-09-11T05:22:02.529115+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9760768,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17577,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17738","reported_seq":17874,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.796639+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:02.796639+0000","last_peered":"2024-09-11T05:22:02.796639+0000","last_clean":"2024-09-11T05:22:02.796639+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:02.796639+0000","last_undegraded":"2024-09-11T05:22:02.796639+0000","last_fullsized":"2024-09-11T05:22:02.796639+0000","mapping_epoch":34,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17571","reported_seq":24923,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:00.833502+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:00.833502+0000","last_peered":"2024-09-11T05:22:00.833502+0000","last_clean":"2024-09-11T05:22:00.833502+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:00.833502+0000","last_undegraded":"2024-09-11T05:22:00.833502+0000","last_fullsized":"2024-09-11T05:22:00.833502+0000","mapping_epoch":46,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10092544,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14115,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17726","reported_seq":17865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.728956+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:02.728956+0000","last_peered":"2024-09-11T05:22:02.728956+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:02.728956+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10141696,"num_objects":2476,"num_object_clones":0,"num_object_copies":4952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14891,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1100}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17421","reported_seq":17551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.807261+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:02.807261+0000","last_peered":"2024-09-11T05:22:02.807261+0000","last_clean":"2024-09-11T05:22:02.807261+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:02.807261+0000","last_undegraded":"2024-09-11T05:22:02.807261+0000","last_fullsized":"2024-09-11T05:22:02.807261+0000","mapping_epoch":34,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10088448,"num_objects":2463,"num_object_clones":0,"num_object_copies":4926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14724,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17556","reported_seq":17694,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.547782+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:02.547782+0000","last_peered":"2024-09-11T05:22:02.547782+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:02.547782+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9891840,"num_objects":2415,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15624,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17232","reported_seq":17403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.766107+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:02.766107+0000","last_peered":"2024-09-11T05:22:02.766107+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:02.766107+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9994240,"num_objects":2440,"num_object_clones":0,"num_object_copies":4880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1002}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17588","reported_seq":17712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.553296+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:02.553296+0000","last_peered":"2024-09-11T05:22:02.553296+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:02.553296+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10014720,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1077}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17277","reported_seq":24369,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.546563+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:02.546563+0000","last_peered":"2024-09-11T05:22:02.546563+0000","last_clean":"2024-09-11T05:22:02.546563+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:02.546563+0000","last_undegraded":"2024-09-11T05:22:02.546563+0000","last_fullsized":"2024-09-11T05:22:02.546563+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17732","reported_seq":17867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.814089+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:02.814089+0000","last_peered":"2024-09-11T05:22:02.814089+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:02.814089+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9879552,"num_objects":2412,"num_object_clones":0,"num_object_copies":4824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14765,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1211}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17239","reported_seq":17379,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.788547+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:02.788547+0000","last_peered":"2024-09-11T05:22:02.788547+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:02.788547+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9891840,"num_objects":2415,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17379","reported_seq":17520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.823353+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:02.823353+0000","last_peered":"2024-09-11T05:22:02.823353+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:02.823353+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9916416,"num_objects":2421,"num_object_clones":0,"num_object_copies":4842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14137,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1123}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17217","reported_seq":17367,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.757718+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:02.757718+0000","last_peered":"2024-09-11T05:22:02.757718+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:02.757718+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9720425,"num_objects":2374,"num_object_clones":0,"num_object_copies":4748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14224,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":158769793,"num_objects":38764,"num_object_clones":0,"num_object_copies":77528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242931,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10241,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":493887488,"data_stored":493809764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512800,"kb_used_data":192336,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93859040,"statfs":{"total":96636764160,"available":96111656960,"internally_reserved":0,"allocated":196952064,"data_stored":196682693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499416,"kb_used_data":120644,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93872424,"statfs":{"total":96636764160,"available":96125362176,"internally_reserved":0,"allocated":123539456,"data_stored":123206865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387837611},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215218,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592088,"kb_used_data":173128,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93779752,"statfs":{"total":96636764160,"available":96030466048,"internally_reserved":0,"allocated":177283072,"data_stored":177013696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":176279552,"data_stored":176276073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":121806848,"data_stored":121740154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195801088,"data_stored":195793537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:03.846 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:04.706 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:04.708 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:04.781 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-11T05:22:03.617634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":158769812,"num_objects":38766,"num_object_clones":0,"num_object_copies":77532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242933,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10242,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1604304,"kb_used_data":486108,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281511216,"statfs":{"total":289910292480,"available":288267485184,"internally_reserved":0,"allocated":497774592,"data_stored":496903254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1144966145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,3,12,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27824128,"num_objects":-6793,"num_object_clones":0,"num_object_copies":-13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6793,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008624"},"pg_stats":[{"pgid":"2.f","version":"47'17754","reported_seq":17837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.826740+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:02.826740+0000","last_peered":"2024-09-11T05:22:02.826740+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:02.826740+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9977856,"num_objects":2436,"num_object_clones":0,"num_object_copies":4872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17754,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17527","reported_seq":17635,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.553339+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:02.553339+0000","last_peered":"2024-09-11T05:22:02.553339+0000","last_clean":"2024-09-11T05:22:02.553339+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:02.553339+0000","last_undegraded":"2024-09-11T05:22:02.553339+0000","last_fullsized":"2024-09-11T05:22:02.553339+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15925,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17333","reported_seq":23782,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.554358+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:02.554358+0000","last_peered":"2024-09-11T05:22:02.554358+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:02.554358+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764888,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15736,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3158,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2048}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17577","reported_seq":17654,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.529115+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:02.529115+0000","last_peered":"2024-09-11T05:22:02.529115+0000","last_clean":"2024-09-11T05:22:02.529115+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:02.529115+0000","last_undegraded":"2024-09-11T05:22:02.529115+0000","last_fullsized":"2024-09-11T05:22:02.529115+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9760768,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17577,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17738","reported_seq":17874,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.796639+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:02.796639+0000","last_peered":"2024-09-11T05:22:02.796639+0000","last_clean":"2024-09-11T05:22:02.796639+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:02.796639+0000","last_undegraded":"2024-09-11T05:22:02.796639+0000","last_fullsized":"2024-09-11T05:22:02.796639+0000","mapping_epoch":34,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17571","reported_seq":24923,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:00.833502+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:00.833502+0000","last_peered":"2024-09-11T05:22:00.833502+0000","last_clean":"2024-09-11T05:22:00.833502+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:00.833502+0000","last_undegraded":"2024-09-11T05:22:00.833502+0000","last_fullsized":"2024-09-11T05:22:00.833502+0000","mapping_epoch":46,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10092544,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14115,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17726","reported_seq":17865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.728956+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:02.728956+0000","last_peered":"2024-09-11T05:22:02.728956+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:02.728956+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10141696,"num_objects":2476,"num_object_clones":0,"num_object_copies":4952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14891,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1100}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17421","reported_seq":17551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.807261+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:02.807261+0000","last_peered":"2024-09-11T05:22:02.807261+0000","last_clean":"2024-09-11T05:22:02.807261+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:02.807261+0000","last_undegraded":"2024-09-11T05:22:02.807261+0000","last_fullsized":"2024-09-11T05:22:02.807261+0000","mapping_epoch":34,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10088448,"num_objects":2463,"num_object_clones":0,"num_object_copies":4926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14724,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17556","reported_seq":17694,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.547782+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:02.547782+0000","last_peered":"2024-09-11T05:22:02.547782+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:02.547782+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9891840,"num_objects":2415,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15624,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17232","reported_seq":17403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.766107+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:02.766107+0000","last_peered":"2024-09-11T05:22:02.766107+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:02.766107+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9994240,"num_objects":2440,"num_object_clones":0,"num_object_copies":4880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1002}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17588","reported_seq":17712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.553296+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:02.553296+0000","last_peered":"2024-09-11T05:22:02.553296+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:02.553296+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10014720,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1077}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17277","reported_seq":24369,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.546563+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:02.546563+0000","last_peered":"2024-09-11T05:22:02.546563+0000","last_clean":"2024-09-11T05:22:02.546563+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:02.546563+0000","last_undegraded":"2024-09-11T05:22:02.546563+0000","last_fullsized":"2024-09-11T05:22:02.546563+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17732","reported_seq":17867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.814089+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:02.814089+0000","last_peered":"2024-09-11T05:22:02.814089+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:02.814089+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9879552,"num_objects":2412,"num_object_clones":0,"num_object_copies":4824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14765,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1211}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17239","reported_seq":17379,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.788547+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:02.788547+0000","last_peered":"2024-09-11T05:22:02.788547+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:02.788547+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9891840,"num_objects":2415,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17379","reported_seq":17520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.823353+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:02.823353+0000","last_peered":"2024-09-11T05:22:02.823353+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:02.823353+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9916416,"num_objects":2421,"num_object_clones":0,"num_object_copies":4842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14137,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1123}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17217","reported_seq":17367,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.757718+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:02.757718+0000","last_peered":"2024-09-11T05:22:02.757718+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:02.757718+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9720425,"num_objects":2374,"num_object_clones":0,"num_object_copies":4748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14224,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":158769793,"num_objects":38764,"num_object_clones":0,"num_object_copies":77528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242931,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10241,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":493887488,"data_stored":493809764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512800,"kb_used_data":192336,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93859040,"statfs":{"total":96636764160,"available":96111656960,"internally_reserved":0,"allocated":196952064,"data_stored":196682693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499416,"kb_used_data":120644,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93872424,"statfs":{"total":96636764160,"available":96125362176,"internally_reserved":0,"allocated":123539456,"data_stored":123206865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387837611},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215218,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592088,"kb_used_data":173128,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93779752,"statfs":{"total":96636764160,"available":96030466048,"internally_reserved":0,"allocated":177283072,"data_stored":177013696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":176279552,"data_stored":176276073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":121806848,"data_stored":121740154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195801088,"data_stored":195793537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:04.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:04.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17754", 'reported_seq': 17837, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.826740+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:02.826740+0000', 'last_peered': '2024-09-11T05:22:02.826740+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:02.826740+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'7700", 'ondisk_log_start': "45'7700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9977856, 'num_objects': 2436, 'num_object_clones': 0, 'num_object_copies': 4872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17754, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1239}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:04.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:22:04.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17333", 'reported_seq': 23782, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.554358+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:22:02.554358+0000', 'last_peered': '2024-09-11T05:22:02.554358+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:22:02.554358+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9764888, 'num_objects': 2385, 'num_object_clones': 0, 'num_object_copies': 4770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 337, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2385, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15736, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3158, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2048}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:04.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:04.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'17726", 'reported_seq': 17865, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.728956+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:22:02.728956+0000', 'last_peered': '2024-09-11T05:22:02.728956+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:02.728956+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'7700", 'ondisk_log_start': "45'7700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10141696, 'num_objects': 2476, 'num_object_clones': 0, 'num_object_copies': 4952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14891, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1100}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17556", 'reported_seq': 17694, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.547782+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:02.547782+0000', 'last_peered': '2024-09-11T05:22:02.547782+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:02.547782+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9891840, 'num_objects': 2415, 'num_object_clones': 0, 'num_object_copies': 4830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15624, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1110}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17232", 'reported_seq': 17403, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.766107+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:02.766107+0000', 'last_peered': '2024-09-11T05:22:02.766107+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:02.766107+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9994240, 'num_objects': 2440, 'num_object_clones': 0, 'num_object_copies': 4880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13985, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1002}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'17588", 'reported_seq': 17712, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.553296+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:02.553296+0000', 'last_peered': '2024-09-11T05:22:02.553296+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:02.553296+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10014720, 'num_objects': 2445, 'num_object_clones': 0, 'num_object_copies': 4890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15617, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1077}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:04.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17732", 'reported_seq': 17867, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.814089+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:02.814089+0000', 'last_peered': '2024-09-11T05:22:02.814089+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:02.814089+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'7700", 'ondisk_log_start': "45'7700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9879552, 'num_objects': 2412, 'num_object_clones': 0, 'num_object_copies': 4824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2412, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14765, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1211}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:04.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:04.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17239", 'reported_seq': 17379, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.788547+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:02.788547+0000', 'last_peered': '2024-09-11T05:22:02.788547+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:02.788547+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9891840, 'num_objects': 2415, 'num_object_clones': 0, 'num_object_copies': 4830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14105, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1080}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:04.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:04.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17379", 'reported_seq': 17520, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.823353+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:02.823353+0000', 'last_peered': '2024-09-11T05:22:02.823353+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:02.823353+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9916416, 'num_objects': 2421, 'num_object_clones': 0, 'num_object_copies': 4842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14137, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1123}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:04.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:04.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17217", 'reported_seq': 17367, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:02.757718+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:02.757718+0000', 'last_peered': '2024-09-11T05:22:02.757718+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:02.757718+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9720425, 'num_objects': 2374, 'num_object_clones': 0, 'num_object_copies': 4748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2374, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14224, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1141}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:04.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:05.900 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:05.901 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32271/77532 objects degraded (41.623%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":603,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":38766,"data_bytes":158769812,"bytes_used":1642807296,"bytes_avail":288267485184,"bytes_total":289910292480,"degraded_objects":32271,"degraded_total":77532,"degraded_ratio":0.41622813805912395,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:05.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:05.973 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:06.748 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:06.748 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:06.813 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-11T05:22:05.618123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":158769812,"num_objects":38766,"num_object_clones":0,"num_object_copies":77532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242933,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10242,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1604304,"kb_used_data":486108,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281511216,"statfs":{"total":289910292480,"available":288267485184,"internally_reserved":0,"allocated":497774592,"data_stored":496903254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1144966145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,3,12,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18915328,"num_objects":-4618,"num_object_clones":0,"num_object_copies":-9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008658"},"pg_stats":[{"pgid":"2.f","version":"47'17754","reported_seq":17837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.826740+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:02.826740+0000","last_peered":"2024-09-11T05:22:02.826740+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:02.826740+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9977856,"num_objects":2436,"num_object_clones":0,"num_object_copies":4872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17754,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17527","reported_seq":17635,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.553339+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:02.553339+0000","last_peered":"2024-09-11T05:22:02.553339+0000","last_clean":"2024-09-11T05:22:02.553339+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:02.553339+0000","last_undegraded":"2024-09-11T05:22:02.553339+0000","last_fullsized":"2024-09-11T05:22:02.553339+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15925,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17333","reported_seq":23782,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.554358+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:02.554358+0000","last_peered":"2024-09-11T05:22:02.554358+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:02.554358+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764888,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15736,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3158,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2048}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17577","reported_seq":17654,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.529115+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:02.529115+0000","last_peered":"2024-09-11T05:22:02.529115+0000","last_clean":"2024-09-11T05:22:02.529115+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:02.529115+0000","last_undegraded":"2024-09-11T05:22:02.529115+0000","last_fullsized":"2024-09-11T05:22:02.529115+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9760768,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17577,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17738","reported_seq":17874,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.796639+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:02.796639+0000","last_peered":"2024-09-11T05:22:02.796639+0000","last_clean":"2024-09-11T05:22:02.796639+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:02.796639+0000","last_undegraded":"2024-09-11T05:22:02.796639+0000","last_fullsized":"2024-09-11T05:22:02.796639+0000","mapping_epoch":34,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17571","reported_seq":24923,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:00.833502+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:00.833502+0000","last_peered":"2024-09-11T05:22:00.833502+0000","last_clean":"2024-09-11T05:22:00.833502+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:00.833502+0000","last_undegraded":"2024-09-11T05:22:00.833502+0000","last_fullsized":"2024-09-11T05:22:00.833502+0000","mapping_epoch":46,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10092544,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14115,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17726","reported_seq":17865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.728956+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:02.728956+0000","last_peered":"2024-09-11T05:22:02.728956+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:02.728956+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10141696,"num_objects":2476,"num_object_clones":0,"num_object_copies":4952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14891,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1100}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17421","reported_seq":17551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.807261+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:02.807261+0000","last_peered":"2024-09-11T05:22:02.807261+0000","last_clean":"2024-09-11T05:22:02.807261+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:02.807261+0000","last_undegraded":"2024-09-11T05:22:02.807261+0000","last_fullsized":"2024-09-11T05:22:02.807261+0000","mapping_epoch":34,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10088448,"num_objects":2463,"num_object_clones":0,"num_object_copies":4926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14724,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17556","reported_seq":17694,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.547782+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:02.547782+0000","last_peered":"2024-09-11T05:22:02.547782+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:02.547782+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9891840,"num_objects":2415,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15624,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17232","reported_seq":17403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.766107+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:02.766107+0000","last_peered":"2024-09-11T05:22:02.766107+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:02.766107+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9994240,"num_objects":2440,"num_object_clones":0,"num_object_copies":4880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1002}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17588","reported_seq":17712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.553296+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:02.553296+0000","last_peered":"2024-09-11T05:22:02.553296+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:02.553296+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10014720,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1077}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17277","reported_seq":24369,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:02.546563+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:02.546563+0000","last_peered":"2024-09-11T05:22:02.546563+0000","last_clean":"2024-09-11T05:22:02.546563+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:02.546563+0000","last_undegraded":"2024-09-11T05:22:02.546563+0000","last_fullsized":"2024-09-11T05:22:02.546563+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17732","reported_seq":17867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.814089+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:02.814089+0000","last_peered":"2024-09-11T05:22:02.814089+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:02.814089+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9879552,"num_objects":2412,"num_object_clones":0,"num_object_copies":4824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14765,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1211}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17239","reported_seq":17379,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.788547+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:02.788547+0000","last_peered":"2024-09-11T05:22:02.788547+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:02.788547+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9891840,"num_objects":2415,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17379","reported_seq":17520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.823353+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:02.823353+0000","last_peered":"2024-09-11T05:22:02.823353+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:02.823353+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9916416,"num_objects":2421,"num_object_clones":0,"num_object_copies":4842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14137,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1123}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17217","reported_seq":17367,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:02.757718+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:02.757718+0000","last_peered":"2024-09-11T05:22:02.757718+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:02.757718+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7200","ondisk_log_start":"45'7200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9720425,"num_objects":2374,"num_object_clones":0,"num_object_copies":4748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14224,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":158769793,"num_objects":38764,"num_object_clones":0,"num_object_copies":77528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242931,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10241,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":493887488,"data_stored":493809764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512800,"kb_used_data":192336,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93859040,"statfs":{"total":96636764160,"available":96111656960,"internally_reserved":0,"allocated":196952064,"data_stored":196682693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499416,"kb_used_data":120644,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93872424,"statfs":{"total":96636764160,"available":96125362176,"internally_reserved":0,"allocated":123539456,"data_stored":123206865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387837611},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215218,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592088,"kb_used_data":173128,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93779752,"statfs":{"total":96636764160,"available":96030466048,"internally_reserved":0,"allocated":177283072,"data_stored":177013696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":176279552,"data_stored":176276073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":121806848,"data_stored":121740154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195801088,"data_stored":195793537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:09.815 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:10.629 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:10.632 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:10.703 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-11T05:22:09.619020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149541524,"num_objects":36513,"num_object_clones":0,"num_object_copies":73026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245186,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10373,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1591684,"kb_used_data":467920,"kb_used_omap":12,"kb_used_meta":1123699,"kb_avail":281523836,"statfs":{"total":289910292480,"available":288280408064,"internally_reserved":0,"allocated":479150080,"data_stored":478278742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1150667777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,5,8,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24010752,"num_objects":-5862,"num_object_clones":0,"num_object_copies":-11724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5862,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002765"},"pg_stats":[{"pgid":"2.f","version":"47'17899","reported_seq":17983,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.866213+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:07.866213+0000","last_peered":"2024-09-11T05:22:07.866213+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:07.866213+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9383936,"num_objects":2291,"num_object_clones":0,"num_object_copies":4582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17899,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17657","reported_seq":17765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.547058+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:07.547058+0000","last_peered":"2024-09-11T05:22:07.547058+0000","last_clean":"2024-09-11T05:22:07.547058+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:07.547058+0000","last_undegraded":"2024-09-11T05:22:07.547058+0000","last_fullsized":"2024-09-11T05:22:07.547058+0000","mapping_epoch":35,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9187328,"num_objects":2243,"num_object_clones":0,"num_object_copies":4486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17472","reported_seq":24183,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.603449+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:07.603449+0000","last_peered":"2024-09-11T05:22:07.603449+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:07.603449+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9195544,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15875,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3289,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17720","reported_seq":17797,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.514298+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:07.514298+0000","last_peered":"2024-09-11T05:22:07.514298+0000","last_clean":"2024-09-11T05:22:07.514298+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:07.514298+0000","last_undegraded":"2024-09-11T05:22:07.514298+0000","last_fullsized":"2024-09-11T05:22:07.514298+0000","mapping_epoch":35,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9175040,"num_objects":2240,"num_object_clones":0,"num_object_copies":4480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17720,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17880","reported_seq":18016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.734232+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:07.734232+0000","last_peered":"2024-09-11T05:22:07.734232+0000","last_clean":"2024-09-11T05:22:07.734232+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:07.734232+0000","last_undegraded":"2024-09-11T05:22:07.734232+0000","last_fullsized":"2024-09-11T05:22:07.734232+0000","mapping_epoch":34,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9375744,"num_objects":2289,"num_object_clones":0,"num_object_copies":4578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14714,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17714","reported_seq":25066,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:05.768676+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:05.768676+0000","last_peered":"2024-09-11T05:22:05.768676+0000","last_clean":"2024-09-11T05:22:05.768676+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:05.768676+0000","last_undegraded":"2024-09-11T05:22:05.768676+0000","last_fullsized":"2024-09-11T05:22:05.768676+0000","mapping_epoch":46,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9506816,"num_objects":2321,"num_object_clones":0,"num_object_copies":4642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14258,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17862","reported_seq":18001,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.848952+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:07.848952+0000","last_peered":"2024-09-11T05:22:07.848952+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:07.848952+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9584640,"num_objects":2340,"num_object_clones":0,"num_object_copies":4680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15027,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17578","reported_seq":17707,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.858931+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:07.858931+0000","last_peered":"2024-09-11T05:22:07.858931+0000","last_clean":"2024-09-11T05:22:07.858931+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:07.858931+0000","last_undegraded":"2024-09-11T05:22:07.858931+0000","last_fullsized":"2024-09-11T05:22:07.858931+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9445376,"num_objects":2306,"num_object_clones":0,"num_object_copies":4612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17699","reported_seq":17837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.551198+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:07.551198+0000","last_peered":"2024-09-11T05:22:07.551198+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:07.551198+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15767,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17370","reported_seq":17541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.805846+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:07.805846+0000","last_peered":"2024-09-11T05:22:07.805846+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:07.805846+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9428992,"num_objects":2302,"num_object_clones":0,"num_object_copies":4604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14123,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17728","reported_seq":17852,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.598514+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:07.598514+0000","last_peered":"2024-09-11T05:22:07.598514+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:07.598514+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441280,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15757,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17415","reported_seq":24507,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.528623+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:07.528623+0000","last_peered":"2024-09-11T05:22:07.528623+0000","last_clean":"2024-09-11T05:22:07.528623+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:07.528623+0000","last_undegraded":"2024-09-11T05:22:07.528623+0000","last_fullsized":"2024-09-11T05:22:07.528623+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9392128,"num_objects":2293,"num_object_clones":0,"num_object_copies":4586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15318,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17866","reported_seq":18004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.829378+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:07.829378+0000","last_peered":"2024-09-11T05:22:07.829378+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:07.829378+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9330688,"num_objects":2278,"num_object_clones":0,"num_object_copies":4556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14899,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1345}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17358","reported_seq":17498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.773074+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:07.773074+0000","last_peered":"2024-09-11T05:22:07.773074+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:07.773074+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14224,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17545","reported_seq":17686,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.791591+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:07.791591+0000","last_peered":"2024-09-11T05:22:07.791591+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:07.791591+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9236480,"num_objects":2255,"num_object_clones":0,"num_object_copies":4510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14303,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17357","reported_seq":17507,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.848741+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:07.848741+0000","last_peered":"2024-09-11T05:22:07.848741+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:07.848741+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9146985,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149541505,"num_objects":36511,"num_object_clones":0,"num_object_copies":73022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245184,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10372,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":475172864,"data_stored":475095140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512472,"kb_used_data":187848,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93859368,"statfs":{"total":96636764160,"available":96111992832,"internally_reserved":0,"allocated":192356352,"data_stored":192086981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484096,"kb_used_data":113648,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93887744,"statfs":{"total":96636764160,"available":96141049856,"internally_reserved":0,"allocated":116375552,"data_stored":116042961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379317931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215219,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595116,"kb_used_data":166424,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93776724,"statfs":{"total":96636764160,"available":96027365376,"internally_reserved":0,"allocated":170418176,"data_stored":170148800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,0,6,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":169238528,"data_stored":169235049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114765824,"data_stored":114699130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":191168512,"data_stored":191160961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:10.704 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:11.612 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:11.613 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:11.647 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-11T05:22:09.619020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149541524,"num_objects":36513,"num_object_clones":0,"num_object_copies":73026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245186,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10373,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1591684,"kb_used_data":467920,"kb_used_omap":12,"kb_used_meta":1123699,"kb_avail":281523836,"statfs":{"total":289910292480,"available":288280408064,"internally_reserved":0,"allocated":479150080,"data_stored":478278742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1150667777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,5,8,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24010752,"num_objects":-5862,"num_object_clones":0,"num_object_copies":-11724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5862,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002765"},"pg_stats":[{"pgid":"2.f","version":"47'17899","reported_seq":17983,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.866213+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:07.866213+0000","last_peered":"2024-09-11T05:22:07.866213+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:07.866213+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9383936,"num_objects":2291,"num_object_clones":0,"num_object_copies":4582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17899,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17657","reported_seq":17765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.547058+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:07.547058+0000","last_peered":"2024-09-11T05:22:07.547058+0000","last_clean":"2024-09-11T05:22:07.547058+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:07.547058+0000","last_undegraded":"2024-09-11T05:22:07.547058+0000","last_fullsized":"2024-09-11T05:22:07.547058+0000","mapping_epoch":35,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9187328,"num_objects":2243,"num_object_clones":0,"num_object_copies":4486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17472","reported_seq":24183,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.603449+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:07.603449+0000","last_peered":"2024-09-11T05:22:07.603449+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:07.603449+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9195544,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15875,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3289,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17720","reported_seq":17797,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.514298+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:07.514298+0000","last_peered":"2024-09-11T05:22:07.514298+0000","last_clean":"2024-09-11T05:22:07.514298+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:07.514298+0000","last_undegraded":"2024-09-11T05:22:07.514298+0000","last_fullsized":"2024-09-11T05:22:07.514298+0000","mapping_epoch":35,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9175040,"num_objects":2240,"num_object_clones":0,"num_object_copies":4480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17720,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17880","reported_seq":18016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.734232+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:07.734232+0000","last_peered":"2024-09-11T05:22:07.734232+0000","last_clean":"2024-09-11T05:22:07.734232+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:07.734232+0000","last_undegraded":"2024-09-11T05:22:07.734232+0000","last_fullsized":"2024-09-11T05:22:07.734232+0000","mapping_epoch":34,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9375744,"num_objects":2289,"num_object_clones":0,"num_object_copies":4578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14714,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17714","reported_seq":25066,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:05.768676+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:05.768676+0000","last_peered":"2024-09-11T05:22:05.768676+0000","last_clean":"2024-09-11T05:22:05.768676+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:05.768676+0000","last_undegraded":"2024-09-11T05:22:05.768676+0000","last_fullsized":"2024-09-11T05:22:05.768676+0000","mapping_epoch":46,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9506816,"num_objects":2321,"num_object_clones":0,"num_object_copies":4642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14258,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'17862","reported_seq":18001,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.848952+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:07.848952+0000","last_peered":"2024-09-11T05:22:07.848952+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:07.848952+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9584640,"num_objects":2340,"num_object_clones":0,"num_object_copies":4680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15027,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17578","reported_seq":17707,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.858931+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:07.858931+0000","last_peered":"2024-09-11T05:22:07.858931+0000","last_clean":"2024-09-11T05:22:07.858931+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:07.858931+0000","last_undegraded":"2024-09-11T05:22:07.858931+0000","last_fullsized":"2024-09-11T05:22:07.858931+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9445376,"num_objects":2306,"num_object_clones":0,"num_object_copies":4612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17699","reported_seq":17837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.551198+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:07.551198+0000","last_peered":"2024-09-11T05:22:07.551198+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:07.551198+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15767,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17370","reported_seq":17541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.805846+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:07.805846+0000","last_peered":"2024-09-11T05:22:07.805846+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:07.805846+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9428992,"num_objects":2302,"num_object_clones":0,"num_object_copies":4604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14123,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17728","reported_seq":17852,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.598514+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:07.598514+0000","last_peered":"2024-09-11T05:22:07.598514+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:07.598514+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441280,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15757,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1217}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17415","reported_seq":24507,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:07.528623+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:07.528623+0000","last_peered":"2024-09-11T05:22:07.528623+0000","last_clean":"2024-09-11T05:22:07.528623+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:07.528623+0000","last_undegraded":"2024-09-11T05:22:07.528623+0000","last_fullsized":"2024-09-11T05:22:07.528623+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9392128,"num_objects":2293,"num_object_clones":0,"num_object_copies":4586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15318,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17866","reported_seq":18004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.829378+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:07.829378+0000","last_peered":"2024-09-11T05:22:07.829378+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:07.829378+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9330688,"num_objects":2278,"num_object_clones":0,"num_object_copies":4556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14899,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1345}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17358","reported_seq":17498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.773074+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:07.773074+0000","last_peered":"2024-09-11T05:22:07.773074+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:07.773074+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9404416,"num_objects":2296,"num_object_clones":0,"num_object_copies":4592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14224,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17545","reported_seq":17686,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.791591+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:07.791591+0000","last_peered":"2024-09-11T05:22:07.791591+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:07.791591+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9236480,"num_objects":2255,"num_object_clones":0,"num_object_copies":4510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14303,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17357","reported_seq":17507,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:07.848741+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:07.848741+0000","last_peered":"2024-09-11T05:22:07.848741+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:07.848741+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7300","ondisk_log_start":"45'7300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9146985,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149541505,"num_objects":36511,"num_object_clones":0,"num_object_copies":73022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245184,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10372,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":475172864,"data_stored":475095140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512472,"kb_used_data":187848,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93859368,"statfs":{"total":96636764160,"available":96111992832,"internally_reserved":0,"allocated":192356352,"data_stored":192086981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484096,"kb_used_data":113648,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93887744,"statfs":{"total":96636764160,"available":96141049856,"internally_reserved":0,"allocated":116375552,"data_stored":116042961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379317931},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215219,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595116,"kb_used_data":166424,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93776724,"statfs":{"total":96636764160,"available":96027365376,"internally_reserved":0,"allocated":170418176,"data_stored":170148800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,0,6,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":169238528,"data_stored":169235049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114765824,"data_stored":114699130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":191168512,"data_stored":191160961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:11.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:11.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17899", 'reported_seq': 17983, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.866213+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:07.866213+0000', 'last_peered': '2024-09-11T05:22:07.866213+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:07.866213+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'7800", 'ondisk_log_start': "45'7800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9383936, 'num_objects': 2291, 'num_object_clones': 0, 'num_object_copies': 4582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17899, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1384}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:11.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17472", 'reported_seq': 24183, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.603449+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:22:07.603449+0000', 'last_peered': '2024-09-11T05:22:07.603449+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:22:07.603449+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9195544, 'num_objects': 2246, 'num_object_clones': 0, 'num_object_copies': 4492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2246, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15875, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3289, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2040}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'17862", 'reported_seq': 18001, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.848952+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:22:07.848952+0000', 'last_peered': '2024-09-11T05:22:07.848952+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:07.848952+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'7800", 'ondisk_log_start': "45'7800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9584640, 'num_objects': 2340, 'num_object_clones': 0, 'num_object_copies': 4680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15027, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17699", 'reported_seq': 17837, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.551198+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:07.551198+0000', 'last_peered': '2024-09-11T05:22:07.551198+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:07.551198+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'7600", 'ondisk_log_start': "45'7600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9306112, 'num_objects': 2272, 'num_object_clones': 0, 'num_object_copies': 4544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15767, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17370", 'reported_seq': 17541, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.805846+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:07.805846+0000', 'last_peered': '2024-09-11T05:22:07.805846+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:07.805846+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9428992, 'num_objects': 2302, 'num_object_clones': 0, 'num_object_copies': 4604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14123, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1140}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'17728", 'reported_seq': 17852, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.598514+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:07.598514+0000', 'last_peered': '2024-09-11T05:22:07.598514+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:07.598514+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'7700", 'ondisk_log_start': "45'7700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9441280, 'num_objects': 2305, 'num_object_clones': 0, 'num_object_copies': 4610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15757, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1217}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17866", 'reported_seq': 18004, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.829378+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:07.829378+0000', 'last_peered': '2024-09-11T05:22:07.829378+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:07.829378+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'7800", 'ondisk_log_start': "45'7800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9330688, 'num_objects': 2278, 'num_object_clones': 0, 'num_object_copies': 4556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2278, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14899, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1345}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:11.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:11.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17358", 'reported_seq': 17498, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.773074+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:07.773074+0000', 'last_peered': '2024-09-11T05:22:07.773074+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:07.773074+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9404416, 'num_objects': 2296, 'num_object_clones': 0, 'num_object_copies': 4592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14224, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1199}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:11.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:11.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17545", 'reported_seq': 17686, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.791591+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:07.791591+0000', 'last_peered': '2024-09-11T05:22:07.791591+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:07.791591+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9236480, 'num_objects': 2255, 'num_object_clones': 0, 'num_object_copies': 4510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14303, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1289}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:11.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:11.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17357", 'reported_seq': 17507, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:07.848741+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:07.848741+0000', 'last_peered': '2024-09-11T05:22:07.848741+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:07.848741+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9146985, 'num_objects': 2234, 'num_object_clones': 0, 'num_object_copies': 4468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14364, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:11.650 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:12.723 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:12.723 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32140/72730 objects degraded (44.191%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":610,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":36365,"data_bytes":148935316,"bytes_used":1631305728,"bytes_avail":288278986752,"bytes_total":289910292480,"degraded_objects":32140,"degraded_total":72730,"degraded_ratio":0.44190842843393374,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":265,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":392},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:12.776 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:12.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:13.637 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:13.638 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:13.687 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-11T05:22:13.620650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":140083860,"num_objects":34204,"num_object_clones":0,"num_object_copies":68408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34204,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":247495,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10493,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160632,"ondisk_log_size":160632,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1589832,"kb_used_data":449420,"kb_used_omap":12,"kb_used_meta":1140339,"kb_avail":281525688,"statfs":{"total":289910292480,"available":288282304512,"internally_reserved":0,"allocated":460206080,"data_stored":459334742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1167707137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,5,4,13,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27561984,"num_objects":-6729,"num_object_clones":0,"num_object_copies":-13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-385,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003458"},"pg_stats":[{"pgid":"2.f","version":"47'18033","reported_seq":18118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.834171+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:12.834171+0000","last_peered":"2024-09-11T05:22:12.834171+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:12.834171+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18033,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17802","reported_seq":17910,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.554828+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:12.554828+0000","last_peered":"2024-09-11T05:22:12.554828+0000","last_clean":"2024-09-11T05:22:12.554828+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:12.554828+0000","last_undegraded":"2024-09-11T05:22:12.554828+0000","last_fullsized":"2024-09-11T05:22:12.554828+0000","mapping_epoch":35,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8593408,"num_objects":2098,"num_object_clones":0,"num_object_copies":4196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16200,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17611","reported_seq":24562,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.593603+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:12.593603+0000","last_peered":"2024-09-11T05:22:12.593603+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:12.593603+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8626200,"num_objects":2107,"num_object_clones":0,"num_object_copies":4214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16014,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3409,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2021}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17855","reported_seq":17932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.504284+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:12.504284+0000","last_peered":"2024-09-11T05:22:12.504284+0000","last_clean":"2024-09-11T05:22:12.504284+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:12.504284+0000","last_undegraded":"2024-09-11T05:22:12.504284+0000","last_fullsized":"2024-09-11T05:22:12.504284+0000","mapping_epoch":35,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8622080,"num_objects":2105,"num_object_clones":0,"num_object_copies":4210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17855,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18027","reported_seq":18163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.856308+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:12.856308+0000","last_peered":"2024-09-11T05:22:12.856308+0000","last_clean":"2024-09-11T05:22:12.856308+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:12.856308+0000","last_undegraded":"2024-09-11T05:22:12.856308+0000","last_fullsized":"2024-09-11T05:22:12.856308+0000","mapping_epoch":34,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17862","reported_seq":25214,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:10.896816+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:10.896816+0000","last_peered":"2024-09-11T05:22:10.896816+0000","last_clean":"2024-09-11T05:22:10.896816+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:10.896816+0000","last_undegraded":"2024-09-11T05:22:10.896816+0000","last_fullsized":"2024-09-11T05:22:10.896816+0000","mapping_epoch":46,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8900608,"num_objects":2173,"num_object_clones":0,"num_object_copies":4346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14406,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'18005","reported_seq":18144,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.844731+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:12.844731+0000","last_peered":"2024-09-11T05:22:12.844731+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:12.844731+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8998912,"num_objects":2197,"num_object_clones":0,"num_object_copies":4394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17727","reported_seq":17857,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.820942+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:12.820942+0000","last_peered":"2024-09-11T05:22:12.820942+0000","last_clean":"2024-09-11T05:22:12.820942+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:12.820942+0000","last_undegraded":"2024-09-11T05:22:12.820942+0000","last_fullsized":"2024-09-11T05:22:12.820942+0000","mapping_epoch":34,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15030,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707557+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:19:25.707557+0000","last_peered":"2024-09-11T05:19:25.707557+0000","last_clean":"2024-09-11T05:19:25.707557+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:19:25.707557+0000","last_undegraded":"2024-09-11T05:19:25.707557+0000","last_fullsized":"2024-09-11T05:19:25.707557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17849","reported_seq":17986,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.561013+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:12.561013+0000","last_peered":"2024-09-11T05:22:12.561013+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:12.561013+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8691712,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15917,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17507","reported_seq":17678,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.874335+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:12.874335+0000","last_peered":"2024-09-11T05:22:12.874335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:12.874335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1277}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17875","reported_seq":17997,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.478667+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:12.478667+0000","last_peered":"2024-09-11T05:22:12.478667+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:12.478667+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8839168,"num_objects":2158,"num_object_clones":0,"num_object_copies":4316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15904,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17564","reported_seq":24656,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.561620+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:12.561620+0000","last_peered":"2024-09-11T05:22:12.561620+0000","last_clean":"2024-09-11T05:22:12.561620+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:12.561620+0000","last_undegraded":"2024-09-11T05:22:12.561620+0000","last_fullsized":"2024-09-11T05:22:12.561620+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8781824,"num_objects":2144,"num_object_clones":0,"num_object_copies":4288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15467,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18020","reported_seq":18158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.792235+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:12.792235+0000","last_peered":"2024-09-11T05:22:12.792235+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:12.792235+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8699904,"num_objects":2124,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15053,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17528","reported_seq":17668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.901058+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:12.901058+0000","last_peered":"2024-09-11T05:22:12.901058+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:12.901058+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14394,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17673","reported_seq":17814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.759817+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:12.759817+0000","last_peered":"2024-09-11T05:22:12.759817+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:12.759817+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712192,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14431,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17491","reported_seq":17640,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.892274+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:12.892274+0000","last_peered":"2024-09-11T05:22:12.892274+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:12.892274+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8598121,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14498,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":140083841,"num_objects":34202,"num_object_clones":0,"num_object_copies":68404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":247493,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10492,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":456269824,"data_stored":456192100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512068,"kb_used_data":183284,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93859772,"statfs":{"total":96636764160,"available":96112406528,"internally_reserved":0,"allocated":187682816,"data_stored":187413445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485484,"kb_used_data":106704,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93886356,"statfs":{"total":96636764160,"available":96139628544,"internally_reserved":0,"allocated":109264896,"data_stored":108932305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387837611},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,4,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592280,"kb_used_data":159432,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93779560,"statfs":{"total":96636764160,"available":96030269440,"internally_reserved":0,"allocated":163258368,"data_stored":162988992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":162287616,"data_stored":162284137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":107589632,"data_stored":107522938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":186392576,"data_stored":186385025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:16.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:17.305 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:17.306 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:17.345 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-09-11T05:22:16.732867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":139506324,"num_objects":34063,"num_object_clones":0,"num_object_copies":68126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":247636,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10493,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160573,"ondisk_log_size":160573,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1604048,"kb_used_data":441440,"kb_used_omap":12,"kb_used_meta":1162547,"kb_avail":281511472,"statfs":{"total":289910292480,"available":288267747328,"internally_reserved":0,"allocated":452034560,"data_stored":451170180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1190448129},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,11,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19263488,"num_objects":-4703,"num_object_clones":0,"num_object_copies":-9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4703,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.114744"},"pg_stats":[{"pgid":"2.f","version":"47'18033","reported_seq":18118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.834171+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:12.834171+0000","last_peered":"2024-09-11T05:22:12.834171+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:12.834171+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18033,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17802","reported_seq":17910,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.554828+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:12.554828+0000","last_peered":"2024-09-11T05:22:12.554828+0000","last_clean":"2024-09-11T05:22:12.554828+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:12.554828+0000","last_undegraded":"2024-09-11T05:22:12.554828+0000","last_fullsized":"2024-09-11T05:22:12.554828+0000","mapping_epoch":35,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8593408,"num_objects":2098,"num_object_clones":0,"num_object_copies":4196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16200,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17611","reported_seq":24562,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.593603+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:12.593603+0000","last_peered":"2024-09-11T05:22:12.593603+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:12.593603+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8626200,"num_objects":2107,"num_object_clones":0,"num_object_copies":4214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16014,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3409,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2021}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17855","reported_seq":17932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.504284+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:12.504284+0000","last_peered":"2024-09-11T05:22:12.504284+0000","last_clean":"2024-09-11T05:22:12.504284+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:12.504284+0000","last_undegraded":"2024-09-11T05:22:12.504284+0000","last_fullsized":"2024-09-11T05:22:12.504284+0000","mapping_epoch":35,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8622080,"num_objects":2105,"num_object_clones":0,"num_object_copies":4210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17855,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18027","reported_seq":18163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.856308+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:12.856308+0000","last_peered":"2024-09-11T05:22:12.856308+0000","last_clean":"2024-09-11T05:22:12.856308+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:12.856308+0000","last_undegraded":"2024-09-11T05:22:12.856308+0000","last_fullsized":"2024-09-11T05:22:12.856308+0000","mapping_epoch":34,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18003","reported_seq":25354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.042898+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:16.042898+0000","last_peered":"2024-09-11T05:22:16.042898+0000","last_clean":"2024-09-11T05:22:16.042898+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:16.042898+0000","last_undegraded":"2024-09-11T05:22:16.042898+0000","last_fullsized":"2024-09-11T05:22:16.042898+0000","mapping_epoch":46,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14547,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'18005","reported_seq":18144,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.844731+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:12.844731+0000","last_peered":"2024-09-11T05:22:12.844731+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:12.844731+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8998912,"num_objects":2197,"num_object_clones":0,"num_object_copies":4394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17727","reported_seq":17857,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.820942+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:12.820942+0000","last_peered":"2024-09-11T05:22:12.820942+0000","last_clean":"2024-09-11T05:22:12.820942+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:12.820942+0000","last_undegraded":"2024-09-11T05:22:12.820942+0000","last_fullsized":"2024-09-11T05:22:12.820942+0000","mapping_epoch":34,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15030,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:14.403473+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:20:14.403473+0000","last_peered":"2024-09-11T05:20:14.403473+0000","last_clean":"2024-09-11T05:20:14.403473+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:20:14.403473+0000","last_undegraded":"2024-09-11T05:20:14.403473+0000","last_fullsized":"2024-09-11T05:20:14.403473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17849","reported_seq":17986,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.561013+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:12.561013+0000","last_peered":"2024-09-11T05:22:12.561013+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:12.561013+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8691712,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15917,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.907981+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:19:31.907981+0000","last_peered":"2024-09-11T05:19:31.907981+0000","last_clean":"2024-09-11T05:19:31.907981+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:19:31.907981+0000","last_undegraded":"2024-09-11T05:19:31.907981+0000","last_fullsized":"2024-09-11T05:19:31.907981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17507","reported_seq":17678,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.874335+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:12.874335+0000","last_peered":"2024-09-11T05:22:12.874335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:12.874335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1277}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17875","reported_seq":17997,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.478667+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:12.478667+0000","last_peered":"2024-09-11T05:22:12.478667+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:12.478667+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8839168,"num_objects":2158,"num_object_clones":0,"num_object_copies":4316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15904,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17564","reported_seq":24656,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.561620+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:12.561620+0000","last_peered":"2024-09-11T05:22:12.561620+0000","last_clean":"2024-09-11T05:22:12.561620+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:12.561620+0000","last_undegraded":"2024-09-11T05:22:12.561620+0000","last_fullsized":"2024-09-11T05:22:12.561620+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8781824,"num_objects":2144,"num_object_clones":0,"num_object_copies":4288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15467,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18020","reported_seq":18158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.792235+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:12.792235+0000","last_peered":"2024-09-11T05:22:12.792235+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:12.792235+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8699904,"num_objects":2124,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15053,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17528","reported_seq":17668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.901058+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:12.901058+0000","last_peered":"2024-09-11T05:22:12.901058+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:12.901058+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14394,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:31.908978+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:19:31.908978+0000","last_peered":"2024-09-11T05:19:31.908978+0000","last_clean":"2024-09-11T05:19:31.908978+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:19:31.908978+0000","last_undegraded":"2024-09-11T05:19:31.908978+0000","last_fullsized":"2024-09-11T05:19:31.908978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17673","reported_seq":17814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.759817+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:12.759817+0000","last_peered":"2024-09-11T05:22:12.759817+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:12.759817+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712192,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14431,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17491","reported_seq":17640,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.892274+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:12.892274+0000","last_peered":"2024-09-11T05:22:12.892274+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:12.892274+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8598121,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14498,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":139506305,"num_objects":34061,"num_object_clones":0,"num_object_copies":68122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":247634,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10492,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":449245184,"data_stored":449174418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160570,"ondisk_log_size":160570,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512068,"kb_used_data":183284,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93859772,"statfs":{"total":96636764160,"available":96112406528,"internally_reserved":0,"allocated":187682816,"data_stored":187413445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499700,"kb_used_data":98724,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93872140,"statfs":{"total":96636764160,"available":96125071360,"internally_reserved":0,"allocated":101093376,"data_stored":100767743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410578603},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592280,"kb_used_data":159432,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93779560,"statfs":{"total":96636764160,"available":96030269440,"internally_reserved":0,"allocated":163258368,"data_stored":162988992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":162287616,"data_stored":162284137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100564992,"data_stored":100505256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":186392576,"data_stored":186385025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:17.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:17.685 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:17.701 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-11T05:22:17.621502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136659604,"num_objects":33368,"num_object_clones":0,"num_object_copies":66736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248331,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10493,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160568,"ondisk_log_size":160568,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1586784,"kb_used_data":436652,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281528736,"statfs":{"total":289910292480,"available":288285425664,"internally_reserved":0,"allocated":447131648,"data_stored":446262031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1177668609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18112512,"num_objects":-4422,"num_object_clones":0,"num_object_copies":-8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4422,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002938"},"pg_stats":[{"pgid":"2.f","version":"47'18033","reported_seq":18118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.834171+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:12.834171+0000","last_peered":"2024-09-11T05:22:12.834171+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:12.834171+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18033,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17943","reported_seq":18054,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.524305+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:17.524305+0000","last_peered":"2024-09-11T05:22:17.524305+0000","last_clean":"2024-09-11T05:22:17.524305+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:17.524305+0000","last_undegraded":"2024-09-11T05:22:17.524305+0000","last_fullsized":"2024-09-11T05:22:17.524305+0000","mapping_epoch":35,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8015872,"num_objects":1957,"num_object_clones":0,"num_object_copies":3914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16341,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17611","reported_seq":24562,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.593603+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:12.593603+0000","last_peered":"2024-09-11T05:22:12.593603+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:12.593603+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8626200,"num_objects":2107,"num_object_clones":0,"num_object_copies":4214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16014,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3409,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2021}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'17984","reported_seq":18064,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.418721+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:17.418721+0000","last_peered":"2024-09-11T05:22:17.418721+0000","last_clean":"2024-09-11T05:22:17.418721+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:17.418721+0000","last_undegraded":"2024-09-11T05:22:17.418721+0000","last_fullsized":"2024-09-11T05:22:17.418721+0000","mapping_epoch":35,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8093696,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17984,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18027","reported_seq":18163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.856308+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:12.856308+0000","last_peered":"2024-09-11T05:22:12.856308+0000","last_clean":"2024-09-11T05:22:12.856308+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:12.856308+0000","last_undegraded":"2024-09-11T05:22:12.856308+0000","last_fullsized":"2024-09-11T05:22:12.856308+0000","mapping_epoch":34,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18003","reported_seq":25354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.042898+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:16.042898+0000","last_peered":"2024-09-11T05:22:16.042898+0000","last_clean":"2024-09-11T05:22:16.042898+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:16.042898+0000","last_undegraded":"2024-09-11T05:22:16.042898+0000","last_fullsized":"2024-09-11T05:22:16.042898+0000","mapping_epoch":46,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14547,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'18005","reported_seq":18144,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.844731+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:12.844731+0000","last_peered":"2024-09-11T05:22:12.844731+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:12.844731+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8998912,"num_objects":2197,"num_object_clones":0,"num_object_copies":4394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17727","reported_seq":17857,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.820942+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:12.820942+0000","last_peered":"2024-09-11T05:22:12.820942+0000","last_clean":"2024-09-11T05:22:12.820942+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:12.820942+0000","last_undegraded":"2024-09-11T05:22:12.820942+0000","last_fullsized":"2024-09-11T05:22:12.820942+0000","mapping_epoch":34,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15030,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:14.403473+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:20:14.403473+0000","last_peered":"2024-09-11T05:20:14.403473+0000","last_clean":"2024-09-11T05:20:14.403473+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:20:14.403473+0000","last_undegraded":"2024-09-11T05:20:14.403473+0000","last_fullsized":"2024-09-11T05:20:14.403473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17992","reported_seq":18132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.378225+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:17.378225+0000","last_peered":"2024-09-11T05:22:17.378225+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:17.378225+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8105984,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16060,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.756409+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:16.756409+0000","last_peered":"2024-09-11T05:22:16.756409+0000","last_clean":"2024-09-11T05:22:16.756409+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:16.756409+0000","last_undegraded":"2024-09-11T05:22:16.756409+0000","last_fullsized":"2024-09-11T05:22:16.756409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17507","reported_seq":17678,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.874335+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:12.874335+0000","last_peered":"2024-09-11T05:22:12.874335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:12.874335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1277}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'18008","reported_seq":18134,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.436031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:17.436031+0000","last_peered":"2024-09-11T05:22:17.436031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:17.436031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8294400,"num_objects":2025,"num_object_clones":0,"num_object_copies":4050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16037,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1497}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'17713","reported_seq":24807,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.494161+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:17.494161+0000","last_peered":"2024-09-11T05:22:17.494161+0000","last_clean":"2024-09-11T05:22:17.494161+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:17.494161+0000","last_undegraded":"2024-09-11T05:22:17.494161+0000","last_fullsized":"2024-09-11T05:22:17.494161+0000","mapping_epoch":44,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15616,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18020","reported_seq":18158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.792235+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:12.792235+0000","last_peered":"2024-09-11T05:22:12.792235+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:12.792235+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8699904,"num_objects":2124,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15053,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17528","reported_seq":17668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.901058+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:12.901058+0000","last_peered":"2024-09-11T05:22:12.901058+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:12.901058+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14394,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.758498+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:16.758498+0000","last_peered":"2024-09-11T05:22:16.758498+0000","last_clean":"2024-09-11T05:22:16.758498+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:16.758498+0000","last_undegraded":"2024-09-11T05:22:16.758498+0000","last_fullsized":"2024-09-11T05:22:16.758498+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17673","reported_seq":17814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.759817+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:12.759817+0000","last_peered":"2024-09-11T05:22:12.759817+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:12.759817+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712192,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14431,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17491","reported_seq":17640,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.892274+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:12.892274+0000","last_peered":"2024-09-11T05:22:12.892274+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:12.892274+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8598121,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14498,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":136659585,"num_objects":33366,"num_object_clones":0,"num_object_copies":66732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33366,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248329,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10492,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":444719104,"data_stored":444648338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160565,"ondisk_log_size":160565,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494804,"kb_used_data":178496,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93877036,"statfs":{"total":96636764160,"available":96130084864,"internally_reserved":0,"allocated":182779904,"data_stored":182505296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499700,"kb_used_data":98724,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93872140,"statfs":{"total":96636764160,"available":96125071360,"internally_reserved":0,"allocated":101093376,"data_stored":100767743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410578603},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592280,"kb_used_data":159432,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93779560,"statfs":{"total":96636764160,"available":96030269440,"internally_reserved":0,"allocated":163258368,"data_stored":162988992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":162287616,"data_stored":162284137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100564992,"data_stored":100505256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181866496,"data_stored":181858945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:17.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18033", 'reported_seq': 18118, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.834171+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:12.834171+0000', 'last_peered': '2024-09-11T05:22:12.834171+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:12.834171+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'8000", 'ondisk_log_start': "45'8000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8835072, 'num_objects': 2157, 'num_object_clones': 0, 'num_object_copies': 4314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18033, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1518}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17611", 'reported_seq': 24562, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.593603+0000', 'last_change': '2024-09-11T05:19:24.649575+0000', 'last_active': '2024-09-11T05:22:12.593603+0000', 'last_peered': '2024-09-11T05:22:12.593603+0000', 'last_clean': '2024-09-11T05:14:12.102591+0000', 'last_became_active': '2024-09-11T05:14:14.215652+0000', 'last_became_peered': '2024-09-11T05:14:14.215652+0000', 'last_unstale': '2024-09-11T05:22:12.593603+0000', 'last_undegraded': '2024-09-11T05:14:13.802611+0000', 'last_fullsized': '2024-09-11T05:14:13.738303+0000', 'mapping_epoch': 42, 'log_start': "45'7600", 'ondisk_log_start': "45'7600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1632", 'last_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:45.985453+0000', 'objects_scrubbed': 1593, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 695, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8626200, 'num_objects': 2107, 'num_object_clones': 0, 'num_object_copies': 4214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 86, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2107, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16014, 'num_write_kb': 33053, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3409, 'num_bytes_recovered': 1323008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2021}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'18005", 'reported_seq': 18144, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.844731+0000', 'last_change': '2024-09-11T05:14:14.250610+0000', 'last_active': '2024-09-11T05:22:12.844731+0000', 'last_peered': '2024-09-11T05:22:12.844731+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:12.844731+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'8000", 'ondisk_log_start': "45'8000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8998912, 'num_objects': 2197, 'num_object_clones': 0, 'num_object_copies': 4394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2197, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15170, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'17992", 'reported_seq': 18132, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:17.378225+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:17.378225+0000', 'last_peered': '2024-09-11T05:22:17.378225+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:17.378225+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'7900", 'ondisk_log_start': "45'7900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8105984, 'num_objects': 1979, 'num_object_clones': 0, 'num_object_copies': 3958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1979, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16060, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1546}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:17.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17507", 'reported_seq': 17678, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.874335+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:12.874335+0000', 'last_peered': '2024-09-11T05:22:12.874335+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:12.874335+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8867840, 'num_objects': 2165, 'num_object_clones': 0, 'num_object_copies': 4330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14260, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1277}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'18008", 'reported_seq': 18134, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:17.436031+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:17.436031+0000', 'last_peered': '2024-09-11T05:22:17.436031+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:17.436031+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'8000", 'ondisk_log_start': "45'8000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8294400, 'num_objects': 2025, 'num_object_clones': 0, 'num_object_copies': 4050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16037, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1497}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'18020", 'reported_seq': 18158, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.792235+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:12.792235+0000', 'last_peered': '2024-09-11T05:22:12.792235+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:12.792235+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'8000", 'ondisk_log_start': "45'8000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8699904, 'num_objects': 2124, 'num_object_clones': 0, 'num_object_copies': 4248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15053, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1499}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17528", 'reported_seq': 17668, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.901058+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:12.901058+0000', 'last_peered': '2024-09-11T05:22:12.901058+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:12.901058+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8708096, 'num_objects': 2126, 'num_object_clones': 0, 'num_object_copies': 4252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14394, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1369}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17673", 'reported_seq': 17814, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.759817+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:12.759817+0000', 'last_peered': '2024-09-11T05:22:12.759817+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:12.759817+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'7600", 'ondisk_log_start': "45'7600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8712192, 'num_objects': 2127, 'num_object_clones': 0, 'num_object_copies': 4254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14431, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1417}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:17.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17491", 'reported_seq': 17640, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:12.892274+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:12.892274+0000', 'last_peered': '2024-09-11T05:22:12.892274+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:12.892274+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8598121, 'num_objects': 2100, 'num_object_clones': 0, 'num_object_copies': 4200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2100, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14498, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1415}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:17.705 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:18.506 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:18.506 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32020/66736 objects degraded (47.980%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":616,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":33368,"data_bytes":136659604,"bytes_used":1624866816,"bytes_avail":288285425664,"bytes_total":289910292480,"degraded_objects":32020,"degraded_total":66736,"degraded_ratio":0.47980100695276912,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":120,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":442},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:18.551 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:18.551 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:19.282 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:19.283 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:19.350 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-11T05:22:17.748675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136659604,"num_objects":33368,"num_object_clones":0,"num_object_copies":66736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248331,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10493,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160568,"ondisk_log_size":160568,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1586784,"kb_used_data":436652,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281528736,"statfs":{"total":289910292480,"available":288285425664,"internally_reserved":0,"allocated":447131648,"data_stored":446262031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1177668609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12881920,"num_objects":-3145,"num_object_clones":0,"num_object_copies":-6290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3145,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.129655"},"pg_stats":[{"pgid":"2.f","version":"47'18033","reported_seq":18118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.834171+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:12.834171+0000","last_peered":"2024-09-11T05:22:12.834171+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:12.834171+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18033,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17943","reported_seq":18054,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.524305+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:17.524305+0000","last_peered":"2024-09-11T05:22:17.524305+0000","last_clean":"2024-09-11T05:22:17.524305+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:17.524305+0000","last_undegraded":"2024-09-11T05:22:17.524305+0000","last_fullsized":"2024-09-11T05:22:17.524305+0000","mapping_epoch":35,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8015872,"num_objects":1957,"num_object_clones":0,"num_object_copies":3914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16341,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17611","reported_seq":24562,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.593603+0000","last_change":"2024-09-11T05:19:24.649575+0000","last_active":"2024-09-11T05:22:12.593603+0000","last_peered":"2024-09-11T05:22:12.593603+0000","last_clean":"2024-09-11T05:14:12.102591+0000","last_became_active":"2024-09-11T05:14:14.215652+0000","last_became_peered":"2024-09-11T05:14:14.215652+0000","last_unstale":"2024-09-11T05:22:12.593603+0000","last_undegraded":"2024-09-11T05:14:13.802611+0000","last_fullsized":"2024-09-11T05:14:13.738303+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8626200,"num_objects":2107,"num_object_clones":0,"num_object_copies":4214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16014,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3409,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2021}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'17984","reported_seq":18064,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.418721+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:17.418721+0000","last_peered":"2024-09-11T05:22:17.418721+0000","last_clean":"2024-09-11T05:22:17.418721+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:17.418721+0000","last_undegraded":"2024-09-11T05:22:17.418721+0000","last_fullsized":"2024-09-11T05:22:17.418721+0000","mapping_epoch":35,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8093696,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17984,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18027","reported_seq":18163,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.856308+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:12.856308+0000","last_peered":"2024-09-11T05:22:12.856308+0000","last_clean":"2024-09-11T05:22:12.856308+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:12.856308+0000","last_undegraded":"2024-09-11T05:22:12.856308+0000","last_fullsized":"2024-09-11T05:22:12.856308+0000","mapping_epoch":34,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773632,"num_objects":2142,"num_object_clones":0,"num_object_copies":4284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18003","reported_seq":25354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.042898+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:16.042898+0000","last_peered":"2024-09-11T05:22:16.042898+0000","last_clean":"2024-09-11T05:22:16.042898+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:16.042898+0000","last_undegraded":"2024-09-11T05:22:16.042898+0000","last_fullsized":"2024-09-11T05:22:16.042898+0000","mapping_epoch":46,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14547,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'18005","reported_seq":18144,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.844731+0000","last_change":"2024-09-11T05:14:14.250610+0000","last_active":"2024-09-11T05:22:12.844731+0000","last_peered":"2024-09-11T05:22:12.844731+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:12.844731+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8998912,"num_objects":2197,"num_object_clones":0,"num_object_copies":4394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17727","reported_seq":17857,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:22:12.820942+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:12.820942+0000","last_peered":"2024-09-11T05:22:12.820942+0000","last_clean":"2024-09-11T05:22:12.820942+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:12.820942+0000","last_undegraded":"2024-09-11T05:22:12.820942+0000","last_fullsized":"2024-09-11T05:22:12.820942+0000","mapping_epoch":34,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835072,"num_objects":2157,"num_object_clones":0,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15030,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:20:14.403473+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:20:14.403473+0000","last_peered":"2024-09-11T05:20:14.403473+0000","last_clean":"2024-09-11T05:20:14.403473+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:20:14.403473+0000","last_undegraded":"2024-09-11T05:20:14.403473+0000","last_fullsized":"2024-09-11T05:20:14.403473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17992","reported_seq":18132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.378225+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:17.378225+0000","last_peered":"2024-09-11T05:22:17.378225+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:17.378225+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8105984,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16060,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.756409+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:16.756409+0000","last_peered":"2024-09-11T05:22:16.756409+0000","last_clean":"2024-09-11T05:22:16.756409+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:16.756409+0000","last_undegraded":"2024-09-11T05:22:16.756409+0000","last_fullsized":"2024-09-11T05:22:16.756409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17507","reported_seq":17678,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.874335+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:12.874335+0000","last_peered":"2024-09-11T05:22:12.874335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:12.874335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1277}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.710761+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:19:25.710761+0000","last_peered":"2024-09-11T05:19:25.710761+0000","last_clean":"2024-09-11T05:19:25.710761+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:19:25.710761+0000","last_undegraded":"2024-09-11T05:19:25.710761+0000","last_fullsized":"2024-09-11T05:19:25.710761+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'18008","reported_seq":18134,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.436031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:17.436031+0000","last_peered":"2024-09-11T05:22:17.436031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:17.436031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8294400,"num_objects":2025,"num_object_clones":0,"num_object_copies":4050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16037,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1497}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.707890+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:19:25.707890+0000","last_peered":"2024-09-11T05:19:25.707890+0000","last_clean":"2024-09-11T05:19:25.707890+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:19:25.707890+0000","last_undegraded":"2024-09-11T05:19:25.707890+0000","last_fullsized":"2024-09-11T05:19:25.707890+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'17713","reported_seq":24807,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.494161+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:17.494161+0000","last_peered":"2024-09-11T05:22:17.494161+0000","last_clean":"2024-09-11T05:22:17.494161+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:17.494161+0000","last_undegraded":"2024-09-11T05:22:17.494161+0000","last_fullsized":"2024-09-11T05:22:17.494161+0000","mapping_epoch":44,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15616,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18020","reported_seq":18158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.792235+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:12.792235+0000","last_peered":"2024-09-11T05:22:12.792235+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:12.792235+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8699904,"num_objects":2124,"num_object_clones":0,"num_object_copies":4248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15053,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708422+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:19:25.708422+0000","last_peered":"2024-09-11T05:19:25.708422+0000","last_clean":"2024-09-11T05:19:25.708422+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:19:25.708422+0000","last_undegraded":"2024-09-11T05:19:25.708422+0000","last_fullsized":"2024-09-11T05:19:25.708422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17528","reported_seq":17668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.901058+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:12.901058+0000","last_peered":"2024-09-11T05:22:12.901058+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:12.901058+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7500","ondisk_log_start":"45'7500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14394,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.758498+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:16.758498+0000","last_peered":"2024-09-11T05:22:16.758498+0000","last_clean":"2024-09-11T05:22:16.758498+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:16.758498+0000","last_undegraded":"2024-09-11T05:22:16.758498+0000","last_fullsized":"2024-09-11T05:22:16.758498+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17673","reported_seq":17814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.759817+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:12.759817+0000","last_peered":"2024-09-11T05:22:12.759817+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:12.759817+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712192,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14431,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708355+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:19:25.708355+0000","last_peered":"2024-09-11T05:19:25.708355+0000","last_clean":"2024-09-11T05:19:25.708355+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:19:25.708355+0000","last_undegraded":"2024-09-11T05:19:25.708355+0000","last_fullsized":"2024-09-11T05:19:25.708355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T05:19:25.708343+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:19:25.708343+0000","last_peered":"2024-09-11T05:19:25.708343+0000","last_clean":"2024-09-11T05:19:25.708343+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:19:25.708343+0000","last_undegraded":"2024-09-11T05:19:25.708343+0000","last_fullsized":"2024-09-11T05:19:25.708343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17491","reported_seq":17640,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:12.892274+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:12.892274+0000","last_peered":"2024-09-11T05:22:12.892274+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:12.892274+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7400","ondisk_log_start":"45'7400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8598121,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14498,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":136659585,"num_objects":33366,"num_object_clones":0,"num_object_copies":66732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33366,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248329,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10492,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":444719104,"data_stored":444648338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160565,"ondisk_log_size":160565,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494804,"kb_used_data":178496,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93877036,"statfs":{"total":96636764160,"available":96130084864,"internally_reserved":0,"allocated":182779904,"data_stored":182505296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499700,"kb_used_data":98724,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93872140,"statfs":{"total":96636764160,"available":96125071360,"internally_reserved":0,"allocated":101093376,"data_stored":100767743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410578603},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592280,"kb_used_data":159432,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93779560,"statfs":{"total":96636764160,"available":96030269440,"internally_reserved":0,"allocated":163258368,"data_stored":162988992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":162287616,"data_stored":162284137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100564992,"data_stored":100505256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181866496,"data_stored":181858945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:22.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:23.133 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:23.134 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:23.202 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-11T05:22:21.622645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":130491028,"num_objects":31862,"num_object_clones":0,"num_object_copies":63724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249837,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10659,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160674,"ondisk_log_size":160674,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1575208,"kb_used_data":425068,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281540312,"statfs":{"total":289910292480,"available":288297279488,"internally_reserved":0,"allocated":435269632,"data_stored":434384283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1177668609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,4,9,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9592832,"num_objects":-2342,"num_object_clones":0,"num_object_copies":-4684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001994"},"pg_stats":[{"pgid":"2.f","version":"48'18147","reported_seq":18236,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.759745+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:17.759745+0000","last_peered":"2024-09-11T05:22:17.759745+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:17.759745+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8368128,"num_objects":2043,"num_object_clones":0,"num_object_copies":4086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18147,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1632}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17943","reported_seq":18054,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.524305+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:17.524305+0000","last_peered":"2024-09-11T05:22:17.524305+0000","last_clean":"2024-09-11T05:22:17.524305+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:17.524305+0000","last_undegraded":"2024-09-11T05:22:17.524305+0000","last_fullsized":"2024-09-11T05:22:17.524305+0000","mapping_epoch":35,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8015872,"num_objects":1957,"num_object_clones":0,"num_object_copies":3914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16341,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17830","reported_seq":24967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:20.994996+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:20.994996+0000","last_peered":"2024-09-11T05:22:20.994996+0000","last_clean":"2024-09-11T05:22:20.994996+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:20.994996+0000","last_undegraded":"2024-09-11T05:22:20.994996+0000","last_fullsized":"2024-09-11T05:22:20.994996+0000","mapping_epoch":48,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7729176,"num_objects":1888,"num_object_clones":0,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16233,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'17984","reported_seq":18064,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.418721+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:17.418721+0000","last_peered":"2024-09-11T05:22:17.418721+0000","last_clean":"2024-09-11T05:22:17.418721+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:17.418721+0000","last_undegraded":"2024-09-11T05:22:17.418721+0000","last_fullsized":"2024-09-11T05:22:17.418721+0000","mapping_epoch":35,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8093696,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17984,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'18172","reported_seq":18312,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.757828+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:17.757828+0000","last_peered":"2024-09-11T05:22:17.757828+0000","last_clean":"2024-09-11T05:22:17.757828+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:17.757828+0000","last_undegraded":"2024-09-11T05:22:17.757828+0000","last_fullsized":"2024-09-11T05:22:17.757828+0000","mapping_epoch":34,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8179712,"num_objects":1997,"num_object_clones":0,"num_object_copies":3994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15006,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18124","reported_seq":25480,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:21.054576+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:21.054576+0000","last_peered":"2024-09-11T05:22:21.054576+0000","last_clean":"2024-09-11T05:22:21.054576+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:21.054576+0000","last_undegraded":"2024-09-11T05:22:21.054576+0000","last_fullsized":"2024-09-11T05:22:21.054576+0000","mapping_epoch":46,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7827456,"num_objects":1911,"num_object_clones":0,"num_object_copies":3822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14668,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'18104","reported_seq":18408,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.884735+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:17.884735+0000","last_peered":"2024-09-11T05:22:17.884735+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:17.884735+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8593408,"num_objects":2098,"num_object_clones":0,"num_object_copies":4196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'17870","reported_seq":18004,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760902+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:17.760902+0000","last_peered":"2024-09-11T05:22:17.760902+0000","last_clean":"2024-09-11T05:22:17.760902+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:17.760902+0000","last_undegraded":"2024-09-11T05:22:17.760902+0000","last_fullsized":"2024-09-11T05:22:17.760902+0000","mapping_epoch":34,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8249344,"num_objects":2014,"num_object_clones":0,"num_object_copies":4028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15173,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17992","reported_seq":18132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.378225+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:17.378225+0000","last_peered":"2024-09-11T05:22:17.378225+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:17.378225+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8105984,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16060,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.756409+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:16.756409+0000","last_peered":"2024-09-11T05:22:16.756409+0000","last_clean":"2024-09-11T05:22:16.756409+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:16.756409+0000","last_undegraded":"2024-09-11T05:22:16.756409+0000","last_fullsized":"2024-09-11T05:22:16.756409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'17644","reported_seq":17819,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.757812+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:17.757812+0000","last_peered":"2024-09-11T05:22:17.757812+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:17.757812+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8306688,"num_objects":2028,"num_object_clones":0,"num_object_copies":4056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14397,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'18008","reported_seq":18134,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.436031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:17.436031+0000","last_peered":"2024-09-11T05:22:17.436031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:17.436031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8294400,"num_objects":2025,"num_object_clones":0,"num_object_copies":4050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16037,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1497}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'17713","reported_seq":24807,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.494161+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:17.494161+0000","last_peered":"2024-09-11T05:22:17.494161+0000","last_clean":"2024-09-11T05:22:17.494161+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:17.494161+0000","last_undegraded":"2024-09-11T05:22:17.494161+0000","last_fullsized":"2024-09-11T05:22:17.494161+0000","mapping_epoch":44,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15616,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18169","reported_seq":18310,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.893711+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:17.893711+0000","last_peered":"2024-09-11T05:22:17.893711+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:17.893711+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8089600,"num_objects":1975,"num_object_clones":0,"num_object_copies":3950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15202,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'17640","reported_seq":17784,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.758016+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:17.758016+0000","last_peered":"2024-09-11T05:22:17.758016+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:17.758016+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8249344,"num_objects":2014,"num_object_clones":0,"num_object_copies":4028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14506,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T05:22:16.758498+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:16.758498+0000","last_peered":"2024-09-11T05:22:16.758498+0000","last_clean":"2024-09-11T05:22:16.758498+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:16.758498+0000","last_undegraded":"2024-09-11T05:22:16.758498+0000","last_fullsized":"2024-09-11T05:22:16.758498+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17804","reported_seq":17949,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.878104+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:17.878104+0000","last_peered":"2024-09-11T05:22:17.878104+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:17.878104+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8175616,"num_objects":1996,"num_object_clones":0,"num_object_copies":3992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14562,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1548}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17627","reported_seq":17781,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:17.757771+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:17.757771+0000","last_peered":"2024-09-11T05:22:17.757771+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:17.757771+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7600","ondisk_log_start":"45'7600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8041065,"num_objects":1964,"num_object_clones":0,"num_object_copies":3928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14634,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":130491009,"num_objects":31860,"num_object_clones":0,"num_object_copies":63720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249835,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10658,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":432205824,"data_stored":432135058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160671,"ondisk_log_size":160671,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494804,"kb_used_data":178496,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93877036,"statfs":{"total":96636764160,"available":96130084864,"internally_reserved":0,"allocated":182779904,"data_stored":182505296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486428,"kb_used_data":93764,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93885412,"statfs":{"total":96636764160,"available":96138661888,"internally_reserved":0,"allocated":96014336,"data_stored":95678208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":402058923},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215221,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593976,"kb_used_data":152808,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93777864,"statfs":{"total":96636764160,"available":96028532736,"internally_reserved":0,"allocated":156475392,"data_stored":156200779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":451735211},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":155959296,"data_stored":155955817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94380032,"data_stored":94320296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181866496,"data_stored":181858945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:23.203 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:24.033 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:24.034 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:24.100 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-09-11T05:22:23.623097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":122397332,"num_objects":29886,"num_object_clones":0,"num_object_copies":59772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251813,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10749,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160550,"ondisk_log_size":160550,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1556428,"kb_used_data":414612,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281559092,"statfs":{"total":289910292480,"available":288316510208,"internally_reserved":0,"allocated":424562688,"data_stored":423666823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1169148929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,4,7,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17686528,"num_objects":-4318,"num_object_clones":0,"num_object_copies":-8636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4318,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001952"},"pg_stats":[{"pgid":"2.f","version":"49'18306","reported_seq":18395,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.905233+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:22.905233+0000","last_peered":"2024-09-11T05:22:22.905233+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:22.905233+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7716864,"num_objects":1884,"num_object_clones":0,"num_object_copies":3768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18306,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1791}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18089","reported_seq":18202,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.531480+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:22.531480+0000","last_peered":"2024-09-11T05:22:22.531480+0000","last_clean":"2024-09-11T05:22:22.531480+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:22.531480+0000","last_undegraded":"2024-09-11T05:22:22.531480+0000","last_fullsized":"2024-09-11T05:22:22.531480+0000","mapping_epoch":35,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16487,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17830","reported_seq":24967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:20.994996+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:20.994996+0000","last_peered":"2024-09-11T05:22:20.994996+0000","last_clean":"2024-09-11T05:22:20.994996+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:20.994996+0000","last_undegraded":"2024-09-11T05:22:20.994996+0000","last_fullsized":"2024-09-11T05:22:20.994996+0000","mapping_epoch":48,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7729176,"num_objects":1888,"num_object_clones":0,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16233,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18121","reported_seq":18203,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.582756+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:22.582756+0000","last_peered":"2024-09-11T05:22:22.582756+0000","last_clean":"2024-09-11T05:22:22.582756+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:22.582756+0000","last_undegraded":"2024-09-11T05:22:22.582756+0000","last_fullsized":"2024-09-11T05:22:22.582756+0000","mapping_epoch":35,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7532544,"num_objects":1839,"num_object_clones":0,"num_object_copies":3678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18121,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18316","reported_seq":18456,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.830820+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:22.830820+0000","last_peered":"2024-09-11T05:22:22.830820+0000","last_clean":"2024-09-11T05:22:22.830820+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:22.830820+0000","last_undegraded":"2024-09-11T05:22:22.830820+0000","last_fullsized":"2024-09-11T05:22:22.830820+0000","mapping_epoch":34,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7589888,"num_objects":1853,"num_object_clones":0,"num_object_copies":3706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15150,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18124","reported_seq":25480,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:21.054576+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:21.054576+0000","last_peered":"2024-09-11T05:22:21.054576+0000","last_clean":"2024-09-11T05:22:21.054576+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:21.054576+0000","last_undegraded":"2024-09-11T05:22:21.054576+0000","last_fullsized":"2024-09-11T05:22:21.054576+0000","mapping_epoch":46,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7827456,"num_objects":1911,"num_object_clones":0,"num_object_copies":3822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14668,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18238","reported_seq":18722,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.956349+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:22.956349+0000","last_peered":"2024-09-11T05:22:22.956349+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:22.956349+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8044544,"num_objects":1964,"num_object_clones":0,"num_object_copies":3928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15403,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18013","reported_seq":18147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.861266+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:22.861266+0000","last_peered":"2024-09-11T05:22:22.861266+0000","last_clean":"2024-09-11T05:22:22.861266+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:22.861266+0000","last_undegraded":"2024-09-11T05:22:22.861266+0000","last_fullsized":"2024-09-11T05:22:22.861266+0000","mapping_epoch":34,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7663616,"num_objects":1871,"num_object_clones":0,"num_object_copies":3742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15316,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18138","reported_seq":18280,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.465316+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:22.465316+0000","last_peered":"2024-09-11T05:22:22.465316+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:22.465316+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7507968,"num_objects":1833,"num_object_clones":0,"num_object_copies":3666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16206,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17809","reported_seq":17982,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.907872+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:22.907872+0000","last_peered":"2024-09-11T05:22:22.907872+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:22.907872+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7630848,"num_objects":1863,"num_object_clones":0,"num_object_copies":3726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14562,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1579}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18113","reported_seq":18241,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.469141+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:22.469141+0000","last_peered":"2024-09-11T05:22:22.469141+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:22.469141+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7864320,"num_objects":1920,"num_object_clones":0,"num_object_copies":3840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16142,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17839","reported_seq":24935,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.552399+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:22.552399+0000","last_peered":"2024-09-11T05:22:22.552399+0000","last_clean":"2024-09-11T05:22:22.552399+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:22.552399+0000","last_undegraded":"2024-09-11T05:22:22.552399+0000","last_fullsized":"2024-09-11T05:22:22.552399+0000","mapping_epoch":44,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7655424,"num_objects":1869,"num_object_clones":0,"num_object_copies":3738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15742,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18303","reported_seq":18445,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.837734+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:22.837734+0000","last_peered":"2024-09-11T05:22:22.837734+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:22.837734+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7540736,"num_objects":1841,"num_object_clones":0,"num_object_copies":3682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17772","reported_seq":17916,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.903584+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:22.903584+0000","last_peered":"2024-09-11T05:22:22.903584+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:22.903584+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7708672,"num_objects":1882,"num_object_clones":0,"num_object_copies":3764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14638,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17948","reported_seq":18092,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.868506+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:22.868506+0000","last_peered":"2024-09-11T05:22:22.868506+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:22.868506+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7585792,"num_objects":1852,"num_object_clones":0,"num_object_copies":3704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14706,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17788","reported_seq":17942,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.941770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:22.941770+0000","last_peered":"2024-09-11T05:22:22.941770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:22.941770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7381609,"num_objects":1803,"num_object_clones":0,"num_object_copies":3606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14795,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1712}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":122397313,"num_objects":29884,"num_object_clones":0,"num_object_copies":59768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251811,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10748,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":420651008,"data_stored":420580242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160547,"ondisk_log_size":160547,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494900,"kb_used_data":174436,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93876940,"statfs":{"total":96636764160,"available":96129986560,"internally_reserved":0,"allocated":178622464,"data_stored":178342598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486428,"kb_used_data":93764,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93885412,"statfs":{"total":96636764160,"available":96138661888,"internally_reserved":0,"allocated":96014336,"data_stored":95678208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":402058923},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575100,"kb_used_data":146412,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93796740,"statfs":{"total":96636764160,"available":96047861760,"internally_reserved":0,"allocated":149925888,"data_stored":149646017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":148832256,"data_stored":148828777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94380032,"data_stored":94320296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177438720,"data_stored":177431169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:24.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:24.101 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18306", 'reported_seq': 18395, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.905233+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:22.905233+0000', 'last_peered': '2024-09-11T05:22:22.905233+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:22.905233+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7716864, 'num_objects': 1884, 'num_object_clones': 0, 'num_object_copies': 3768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1884, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18306, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1791}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18238", 'reported_seq': 18722, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.956349+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:22:22.956349+0000', 'last_peered': '2024-09-11T05:22:22.956349+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:22.956349+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'8200", 'ondisk_log_start': "45'8200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8044544, 'num_objects': 1964, 'num_object_clones': 0, 'num_object_copies': 3928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15403, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 170, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18138", 'reported_seq': 18280, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.465316+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:22.465316+0000', 'last_peered': '2024-09-11T05:22:22.465316+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:22.465316+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'8100", 'ondisk_log_start': "45'8100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7507968, 'num_objects': 1833, 'num_object_clones': 0, 'num_object_copies': 3666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16206, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1692}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17809", 'reported_seq': 17982, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.907872+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:22.907872+0000', 'last_peered': '2024-09-11T05:22:22.907872+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:22.907872+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'7800", 'ondisk_log_start': "45'7800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7630848, 'num_objects': 1863, 'num_object_clones': 0, 'num_object_copies': 3726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14562, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1579}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18113", 'reported_seq': 18241, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.469141+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:22.469141+0000', 'last_peered': '2024-09-11T05:22:22.469141+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:22.469141+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'8100", 'ondisk_log_start': "45'8100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7864320, 'num_objects': 1920, 'num_object_clones': 0, 'num_object_copies': 3840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16142, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1602}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:24.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:24.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18303", 'reported_seq': 18445, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.837734+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:22.837734+0000', 'last_peered': '2024-09-11T05:22:22.837734+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:22.837734+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7540736, 'num_objects': 1841, 'num_object_clones': 0, 'num_object_copies': 3682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15336, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1782}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:24.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:24.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17772", 'reported_seq': 17916, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.903584+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:22.903584+0000', 'last_peered': '2024-09-11T05:22:22.903584+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:22.903584+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'7700", 'ondisk_log_start': "45'7700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7708672, 'num_objects': 1882, 'num_object_clones': 0, 'num_object_copies': 3764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14638, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1613}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:24.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:24.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17948", 'reported_seq': 18092, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.868506+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:22.868506+0000', 'last_peered': '2024-09-11T05:22:22.868506+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:22.868506+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'7900", 'ondisk_log_start': "45'7900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7585792, 'num_objects': 1852, 'num_object_clones': 0, 'num_object_copies': 3704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1852, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14706, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1692}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:24.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:24.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'17788", 'reported_seq': 17942, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:22.941770+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:22.941770+0000', 'last_peered': '2024-09-11T05:22:22.941770+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:22.941770+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'7700", 'ondisk_log_start': "45'7700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7381609, 'num_objects': 1803, 'num_object_clones': 0, 'num_object_copies': 3606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14795, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1712}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:24.103 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:25.101 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:25.102 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31764/59772 objects degraded (53.142%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":622,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":29886,"data_bytes":122397332,"bytes_used":1593782272,"bytes_avail":288316510208,"bytes_total":289910292480,"degraded_objects":31764,"degraded_total":59772,"degraded_ratio":0.53141939369604496,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":256,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":539},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:25.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:25.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:25.946 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:25.947 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:25.997 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-11T05:22:25.623607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":122397332,"num_objects":29886,"num_object_clones":0,"num_object_copies":59772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251813,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10749,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160550,"ondisk_log_size":160550,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1556428,"kb_used_data":414612,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281559092,"statfs":{"total":289910292480,"available":288316510208,"internally_reserved":0,"allocated":424562688,"data_stored":423666823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1169148929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,4,7,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17108992,"num_objects":-4177,"num_object_clones":0,"num_object_copies":-8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4177,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.890739"},"pg_stats":[{"pgid":"2.f","version":"49'18306","reported_seq":18395,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.905233+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:22.905233+0000","last_peered":"2024-09-11T05:22:22.905233+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:22.905233+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7716864,"num_objects":1884,"num_object_clones":0,"num_object_copies":3768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18306,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1791}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18089","reported_seq":18202,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.531480+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:22.531480+0000","last_peered":"2024-09-11T05:22:22.531480+0000","last_clean":"2024-09-11T05:22:22.531480+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:22.531480+0000","last_undegraded":"2024-09-11T05:22:22.531480+0000","last_fullsized":"2024-09-11T05:22:22.531480+0000","mapping_epoch":35,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16487,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17830","reported_seq":24967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:20.994996+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:20.994996+0000","last_peered":"2024-09-11T05:22:20.994996+0000","last_clean":"2024-09-11T05:22:20.994996+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:20.994996+0000","last_undegraded":"2024-09-11T05:22:20.994996+0000","last_fullsized":"2024-09-11T05:22:20.994996+0000","mapping_epoch":48,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7729176,"num_objects":1888,"num_object_clones":0,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16233,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18121","reported_seq":18203,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.582756+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:22.582756+0000","last_peered":"2024-09-11T05:22:22.582756+0000","last_clean":"2024-09-11T05:22:22.582756+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:22.582756+0000","last_undegraded":"2024-09-11T05:22:22.582756+0000","last_fullsized":"2024-09-11T05:22:22.582756+0000","mapping_epoch":35,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7532544,"num_objects":1839,"num_object_clones":0,"num_object_copies":3678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18121,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18316","reported_seq":18456,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.830820+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:22.830820+0000","last_peered":"2024-09-11T05:22:22.830820+0000","last_clean":"2024-09-11T05:22:22.830820+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:22.830820+0000","last_undegraded":"2024-09-11T05:22:22.830820+0000","last_fullsized":"2024-09-11T05:22:22.830820+0000","mapping_epoch":34,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7589888,"num_objects":1853,"num_object_clones":0,"num_object_copies":3706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15150,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18124","reported_seq":25480,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:21.054576+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:21.054576+0000","last_peered":"2024-09-11T05:22:21.054576+0000","last_clean":"2024-09-11T05:22:21.054576+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:21.054576+0000","last_undegraded":"2024-09-11T05:22:21.054576+0000","last_fullsized":"2024-09-11T05:22:21.054576+0000","mapping_epoch":46,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7827456,"num_objects":1911,"num_object_clones":0,"num_object_copies":3822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14668,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18238","reported_seq":18722,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.956349+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:22.956349+0000","last_peered":"2024-09-11T05:22:22.956349+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:22.956349+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8044544,"num_objects":1964,"num_object_clones":0,"num_object_copies":3928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15403,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18013","reported_seq":18147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.861266+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:22.861266+0000","last_peered":"2024-09-11T05:22:22.861266+0000","last_clean":"2024-09-11T05:22:22.861266+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:22.861266+0000","last_undegraded":"2024-09-11T05:22:22.861266+0000","last_fullsized":"2024-09-11T05:22:22.861266+0000","mapping_epoch":34,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7663616,"num_objects":1871,"num_object_clones":0,"num_object_copies":3742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15316,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18138","reported_seq":18280,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.465316+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:22.465316+0000","last_peered":"2024-09-11T05:22:22.465316+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:22.465316+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7507968,"num_objects":1833,"num_object_clones":0,"num_object_copies":3666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16206,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17809","reported_seq":17982,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.907872+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:22.907872+0000","last_peered":"2024-09-11T05:22:22.907872+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:22.907872+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7630848,"num_objects":1863,"num_object_clones":0,"num_object_copies":3726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14562,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1579}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18113","reported_seq":18241,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.469141+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:22.469141+0000","last_peered":"2024-09-11T05:22:22.469141+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:22.469141+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7864320,"num_objects":1920,"num_object_clones":0,"num_object_copies":3840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16142,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1602}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17839","reported_seq":24935,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:22.552399+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:22.552399+0000","last_peered":"2024-09-11T05:22:22.552399+0000","last_clean":"2024-09-11T05:22:22.552399+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:22.552399+0000","last_undegraded":"2024-09-11T05:22:22.552399+0000","last_fullsized":"2024-09-11T05:22:22.552399+0000","mapping_epoch":44,"log_start":"45'7800","ondisk_log_start":"45'7800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7655424,"num_objects":1869,"num_object_clones":0,"num_object_copies":3738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15742,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18303","reported_seq":18445,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.837734+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:22.837734+0000","last_peered":"2024-09-11T05:22:22.837734+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:22.837734+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7540736,"num_objects":1841,"num_object_clones":0,"num_object_copies":3682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15336,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17772","reported_seq":17916,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.903584+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:22.903584+0000","last_peered":"2024-09-11T05:22:22.903584+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:22.903584+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7708672,"num_objects":1882,"num_object_clones":0,"num_object_copies":3764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14638,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17948","reported_seq":18092,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.868506+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:22.868506+0000","last_peered":"2024-09-11T05:22:22.868506+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:22.868506+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7585792,"num_objects":1852,"num_object_clones":0,"num_object_copies":3704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14706,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17788","reported_seq":17942,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:22.941770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:22.941770+0000","last_peered":"2024-09-11T05:22:22.941770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:22.941770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7700","ondisk_log_start":"45'7700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7381609,"num_objects":1803,"num_object_clones":0,"num_object_copies":3606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14795,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1712}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":122397313,"num_objects":29884,"num_object_clones":0,"num_object_copies":59768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251811,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10748,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":420651008,"data_stored":420580242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160547,"ondisk_log_size":160547,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494900,"kb_used_data":174436,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93876940,"statfs":{"total":96636764160,"available":96129986560,"internally_reserved":0,"allocated":178622464,"data_stored":178342598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486428,"kb_used_data":93764,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93885412,"statfs":{"total":96636764160,"available":96138661888,"internally_reserved":0,"allocated":96014336,"data_stored":95678208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":402058923},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575100,"kb_used_data":146412,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93796740,"statfs":{"total":96636764160,"available":96047861760,"internally_reserved":0,"allocated":149925888,"data_stored":149646017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":148832256,"data_stored":148828777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94380032,"data_stored":94320296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177438720,"data_stored":177431169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:29.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:29.786 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:29.786 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:29.826 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-11T05:22:29.624517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113279636,"num_objects":27660,"num_object_clones":0,"num_object_copies":55320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254039,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10820,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1569092,"kb_used_data":396744,"kb_used_omap":12,"kb_used_meta":1172275,"kb_avail":281546428,"statfs":{"total":289910292480,"available":288303542272,"internally_reserved":0,"allocated":406265856,"data_stored":405369991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1200409601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,3,9,7,5,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23379968,"num_objects":-5708,"num_object_clones":0,"num_object_copies":-11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.875841"},"pg_stats":[{"pgid":"2.f","version":"49'18417","reported_seq":18506,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.940393+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:27.940393+0000","last_peered":"2024-09-11T05:22:27.940393+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:27.940393+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7262208,"num_objects":1773,"num_object_clones":0,"num_object_copies":3546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18417,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1902}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18241","reported_seq":18354,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.603959+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:27.603959+0000","last_peered":"2024-09-11T05:22:27.603959+0000","last_clean":"2024-09-11T05:22:27.603959+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:27.603959+0000","last_undegraded":"2024-09-11T05:22:27.603959+0000","last_fullsized":"2024-09-11T05:22:27.603959+0000","mapping_epoch":35,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16639,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17968","reported_seq":25105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:26.088670+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:26.088670+0000","last_peered":"2024-09-11T05:22:26.088670+0000","last_clean":"2024-09-11T05:22:26.088670+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:26.088670+0000","last_undegraded":"2024-09-11T05:22:26.088670+0000","last_fullsized":"2024-09-11T05:22:26.088670+0000","mapping_epoch":48,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7163928,"num_objects":1750,"num_object_clones":0,"num_object_copies":3500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16371,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18260","reported_seq":18342,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.598545+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:27.598545+0000","last_peered":"2024-09-11T05:22:27.598545+0000","last_clean":"2024-09-11T05:22:27.598545+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:27.598545+0000","last_undegraded":"2024-09-11T05:22:27.598545+0000","last_fullsized":"2024-09-11T05:22:27.598545+0000","mapping_epoch":35,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18260,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18462","reported_seq":18599,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.947982+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:27.947982+0000","last_peered":"2024-09-11T05:22:27.947982+0000","last_clean":"2024-09-11T05:22:27.947982+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:27.947982+0000","last_undegraded":"2024-09-11T05:22:27.947982+0000","last_fullsized":"2024-09-11T05:22:27.947982+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15296,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18243","reported_seq":25599,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:26.047612+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:26.047612+0000","last_peered":"2024-09-11T05:22:26.047612+0000","last_clean":"2024-09-11T05:22:26.047612+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:26.047612+0000","last_undegraded":"2024-09-11T05:22:26.047612+0000","last_fullsized":"2024-09-11T05:22:26.047612+0000","mapping_epoch":46,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14787,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18367","reported_seq":18993,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.915559+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:27.915559+0000","last_peered":"2024-09-11T05:22:27.915559+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:27.915559+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7516160,"num_objects":1835,"num_object_clones":0,"num_object_copies":3670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15532,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18142","reported_seq":18276,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.789446+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:27.789446+0000","last_peered":"2024-09-11T05:22:27.789446+0000","last_clean":"2024-09-11T05:22:27.789446+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:27.789446+0000","last_undegraded":"2024-09-11T05:22:27.789446+0000","last_fullsized":"2024-09-11T05:22:27.789446+0000","mapping_epoch":34,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7135232,"num_objects":1742,"num_object_clones":0,"num_object_copies":3484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18261","reported_seq":18403,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.543672+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:27.543672+0000","last_peered":"2024-09-11T05:22:27.543672+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:27.543672+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7004160,"num_objects":1710,"num_object_clones":0,"num_object_copies":3420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16329,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1815}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17965","reported_seq":18140,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.898516+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:27.898516+0000","last_peered":"2024-09-11T05:22:27.898516+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:27.898516+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18262","reported_seq":18390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.557637+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:27.557637+0000","last_peered":"2024-09-11T05:22:27.557637+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:27.557637+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7254016,"num_objects":1771,"num_object_clones":0,"num_object_copies":3542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1751}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17981","reported_seq":25077,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.617392+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:27.617392+0000","last_peered":"2024-09-11T05:22:27.617392+0000","last_clean":"2024-09-11T05:22:27.617392+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:27.617392+0000","last_undegraded":"2024-09-11T05:22:27.617392+0000","last_fullsized":"2024-09-11T05:22:27.617392+0000","mapping_epoch":44,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7073792,"num_objects":1727,"num_object_clones":0,"num_object_copies":3454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15884,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18453","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.820496+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:27.820496+0000","last_peered":"2024-09-11T05:22:27.820496+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:27.820496+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15486,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17916","reported_seq":18060,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.865569+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:27.865569+0000","last_peered":"2024-09-11T05:22:27.865569+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:27.865569+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7118848,"num_objects":1738,"num_object_clones":0,"num_object_copies":3476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14782,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18103","reported_seq":18248,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.937085+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:27.937085+0000","last_peered":"2024-09-11T05:22:27.937085+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:27.937085+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17932","reported_seq":18086,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.896378+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:27.896378+0000","last_peered":"2024-09-11T05:22:27.896378+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:27.896378+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6791785,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14939,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":113279617,"num_objects":27658,"num_object_clones":0,"num_object_copies":55316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254037,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10819,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":402186240,"data_stored":402115474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494648,"kb_used_data":170024,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93877192,"statfs":{"total":96636764160,"available":96130244608,"internally_reserved":0,"allocated":174104576,"data_stored":173824710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501852,"kb_used_data":86984,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93869988,"statfs":{"total":96636764160,"available":96122867712,"internally_reserved":0,"allocated":89071616,"data_stored":88735488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424799915},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,3,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572592,"kb_used_data":139736,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93799248,"statfs":{"total":96636764160,"available":96050429952,"internally_reserved":0,"allocated":143089664,"data_stored":142809793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":141893632,"data_stored":141890153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87457792,"data_stored":87398056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172834816,"data_stored":172827265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:29.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:30.776 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:30.777 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:30.871 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-11T05:22:29.624517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":113279636,"num_objects":27660,"num_object_clones":0,"num_object_copies":55320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254039,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10820,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1569092,"kb_used_data":396744,"kb_used_omap":12,"kb_used_meta":1172275,"kb_avail":281546428,"statfs":{"total":289910292480,"available":288303542272,"internally_reserved":0,"allocated":406265856,"data_stored":405369991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1200409601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,3,9,7,5,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23379968,"num_objects":-5708,"num_object_clones":0,"num_object_copies":-11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.875841"},"pg_stats":[{"pgid":"2.f","version":"49'18417","reported_seq":18506,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.940393+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:27.940393+0000","last_peered":"2024-09-11T05:22:27.940393+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:27.940393+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7262208,"num_objects":1773,"num_object_clones":0,"num_object_copies":3546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18417,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1902}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18241","reported_seq":18354,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.603959+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:27.603959+0000","last_peered":"2024-09-11T05:22:27.603959+0000","last_clean":"2024-09-11T05:22:27.603959+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:27.603959+0000","last_undegraded":"2024-09-11T05:22:27.603959+0000","last_fullsized":"2024-09-11T05:22:27.603959+0000","mapping_epoch":35,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16639,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17968","reported_seq":25105,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:26.088670+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:26.088670+0000","last_peered":"2024-09-11T05:22:26.088670+0000","last_clean":"2024-09-11T05:22:26.088670+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:26.088670+0000","last_undegraded":"2024-09-11T05:22:26.088670+0000","last_fullsized":"2024-09-11T05:22:26.088670+0000","mapping_epoch":48,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7163928,"num_objects":1750,"num_object_clones":0,"num_object_copies":3500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16371,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18260","reported_seq":18342,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.598545+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:27.598545+0000","last_peered":"2024-09-11T05:22:27.598545+0000","last_clean":"2024-09-11T05:22:27.598545+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:27.598545+0000","last_undegraded":"2024-09-11T05:22:27.598545+0000","last_fullsized":"2024-09-11T05:22:27.598545+0000","mapping_epoch":35,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18260,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18462","reported_seq":18599,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.947982+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:27.947982+0000","last_peered":"2024-09-11T05:22:27.947982+0000","last_clean":"2024-09-11T05:22:27.947982+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:27.947982+0000","last_undegraded":"2024-09-11T05:22:27.947982+0000","last_fullsized":"2024-09-11T05:22:27.947982+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15296,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18243","reported_seq":25599,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:26.047612+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:26.047612+0000","last_peered":"2024-09-11T05:22:26.047612+0000","last_clean":"2024-09-11T05:22:26.047612+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:26.047612+0000","last_undegraded":"2024-09-11T05:22:26.047612+0000","last_fullsized":"2024-09-11T05:22:26.047612+0000","mapping_epoch":46,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14787,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18367","reported_seq":18993,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.915559+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:27.915559+0000","last_peered":"2024-09-11T05:22:27.915559+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:27.915559+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7516160,"num_objects":1835,"num_object_clones":0,"num_object_copies":3670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15532,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18142","reported_seq":18276,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.789446+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:27.789446+0000","last_peered":"2024-09-11T05:22:27.789446+0000","last_clean":"2024-09-11T05:22:27.789446+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:27.789446+0000","last_undegraded":"2024-09-11T05:22:27.789446+0000","last_fullsized":"2024-09-11T05:22:27.789446+0000","mapping_epoch":34,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7135232,"num_objects":1742,"num_object_clones":0,"num_object_copies":3484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18261","reported_seq":18403,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.543672+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:27.543672+0000","last_peered":"2024-09-11T05:22:27.543672+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:27.543672+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7004160,"num_objects":1710,"num_object_clones":0,"num_object_copies":3420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16329,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1815}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17965","reported_seq":18140,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.898516+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:27.898516+0000","last_peered":"2024-09-11T05:22:27.898516+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:27.898516+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18262","reported_seq":18390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.557637+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:27.557637+0000","last_peered":"2024-09-11T05:22:27.557637+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:27.557637+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7254016,"num_objects":1771,"num_object_clones":0,"num_object_copies":3542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1751}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17981","reported_seq":25077,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.617392+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:27.617392+0000","last_peered":"2024-09-11T05:22:27.617392+0000","last_clean":"2024-09-11T05:22:27.617392+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:27.617392+0000","last_undegraded":"2024-09-11T05:22:27.617392+0000","last_fullsized":"2024-09-11T05:22:27.617392+0000","mapping_epoch":44,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7073792,"num_objects":1727,"num_object_clones":0,"num_object_copies":3454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15884,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18453","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.820496+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:27.820496+0000","last_peered":"2024-09-11T05:22:27.820496+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:27.820496+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15486,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17916","reported_seq":18060,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.865569+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:27.865569+0000","last_peered":"2024-09-11T05:22:27.865569+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:27.865569+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7118848,"num_objects":1738,"num_object_clones":0,"num_object_copies":3476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14782,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18103","reported_seq":18248,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.937085+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:27.937085+0000","last_peered":"2024-09-11T05:22:27.937085+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:27.937085+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17932","reported_seq":18086,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.896378+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:27.896378+0000","last_peered":"2024-09-11T05:22:27.896378+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:27.896378+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6791785,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14939,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":113279617,"num_objects":27658,"num_object_clones":0,"num_object_copies":55316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254037,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10819,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":402186240,"data_stored":402115474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494648,"kb_used_data":170024,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93877192,"statfs":{"total":96636764160,"available":96130244608,"internally_reserved":0,"allocated":174104576,"data_stored":173824710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501852,"kb_used_data":86984,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93869988,"statfs":{"total":96636764160,"available":96122867712,"internally_reserved":0,"allocated":89071616,"data_stored":88735488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424799915},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,3,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572592,"kb_used_data":139736,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93799248,"statfs":{"total":96636764160,"available":96050429952,"internally_reserved":0,"allocated":143089664,"data_stored":142809793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":141893632,"data_stored":141890153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87457792,"data_stored":87398056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172834816,"data_stored":172827265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:30.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18417", 'reported_seq': 18506, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.940393+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:27.940393+0000', 'last_peered': '2024-09-11T05:22:27.940393+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:27.940393+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7262208, 'num_objects': 1773, 'num_object_clones': 0, 'num_object_copies': 3546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18417, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1902}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18367", 'reported_seq': 18993, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.915559+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:22:27.915559+0000', 'last_peered': '2024-09-11T05:22:27.915559+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:27.915559+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7516160, 'num_objects': 1835, 'num_object_clones': 0, 'num_object_copies': 3670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3337, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15532, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 241, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18261", 'reported_seq': 18403, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.543672+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:27.543672+0000', 'last_peered': '2024-09-11T05:22:27.543672+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:27.543672+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'8200", 'ondisk_log_start': "45'8200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7004160, 'num_objects': 1710, 'num_object_clones': 0, 'num_object_copies': 3420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16329, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1815}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17965", 'reported_seq': 18140, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.898516+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:27.898516+0000', 'last_peered': '2024-09-11T05:22:27.898516+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:27.898516+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'7900", 'ondisk_log_start': "45'7900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6991872, 'num_objects': 1707, 'num_object_clones': 0, 'num_object_copies': 3414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14718, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1735}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:30.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18262", 'reported_seq': 18390, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.557637+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:27.557637+0000', 'last_peered': '2024-09-11T05:22:27.557637+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:27.557637+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'8200", 'ondisk_log_start': "45'8200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7254016, 'num_objects': 1771, 'num_object_clones': 0, 'num_object_copies': 3542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16291, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1751}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18453", 'reported_seq': 18595, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.820496+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:27.820496+0000', 'last_peered': '2024-09-11T05:22:27.820496+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:27.820496+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6926336, 'num_objects': 1691, 'num_object_clones': 0, 'num_object_copies': 3382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15486, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1932}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'17916", 'reported_seq': 18060, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.865569+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:27.865569+0000', 'last_peered': '2024-09-11T05:22:27.865569+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:27.865569+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'7900", 'ondisk_log_start': "45'7900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7118848, 'num_objects': 1738, 'num_object_clones': 0, 'num_object_copies': 3476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1738, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14782, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1757}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18103", 'reported_seq': 18248, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.937085+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:27.937085+0000', 'last_peered': '2024-09-11T05:22:27.937085+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:27.937085+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'8100", 'ondisk_log_start': "45'8100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6950912, 'num_objects': 1697, 'num_object_clones': 0, 'num_object_copies': 3394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14861, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1847}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:30.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'17932", 'reported_seq': 18086, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:27.896378+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:27.896378+0000', 'last_peered': '2024-09-11T05:22:27.896378+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:27.896378+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'7900", 'ondisk_log_start': "45'7900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6791785, 'num_objects': 1659, 'num_object_clones': 0, 'num_object_copies': 3318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14939, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1856}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:30.875 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:31.813 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:31.813 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31693/54756 objects degraded (57.880%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":629,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":27378,"data_bytes":112124564,"bytes_used":1609641984,"bytes_avail":288300650496,"bytes_total":289910292480,"degraded_objects":31693,"degraded_total":54756,"degraded_ratio":0.57880414931696988,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:31.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:31.873 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:32.641 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:32.642 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:32.699 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-11T05:22:31.625024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":112124564,"num_objects":27378,"num_object_clones":0,"num_object_copies":54756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27378,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254321,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10820,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1571916,"kb_used_data":389840,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281543604,"statfs":{"total":289910292480,"available":288300650496,"internally_reserved":0,"allocated":399196160,"data_stored":398300295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1210371073},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,1,5,8,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19759104,"num_objects":-4824,"num_object_clones":0,"num_object_copies":-9648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002846"},"pg_stats":[{"pgid":"2.f","version":"49'18417","reported_seq":18506,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.940393+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:27.940393+0000","last_peered":"2024-09-11T05:22:27.940393+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:27.940393+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7262208,"num_objects":1773,"num_object_clones":0,"num_object_copies":3546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18417,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1902}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18241","reported_seq":18354,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.603959+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:27.603959+0000","last_peered":"2024-09-11T05:22:27.603959+0000","last_clean":"2024-09-11T05:22:27.603959+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:27.603959+0000","last_undegraded":"2024-09-11T05:22:27.603959+0000","last_fullsized":"2024-09-11T05:22:27.603959+0000","mapping_epoch":35,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16639,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18104","reported_seq":25241,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:31.085369+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:31.085369+0000","last_peered":"2024-09-11T05:22:31.085369+0000","last_clean":"2024-09-11T05:22:31.085369+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:31.085369+0000","last_undegraded":"2024-09-11T05:22:31.085369+0000","last_fullsized":"2024-09-11T05:22:31.085369+0000","mapping_epoch":48,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6606872,"num_objects":1614,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16507,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18260","reported_seq":18342,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.598545+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:27.598545+0000","last_peered":"2024-09-11T05:22:27.598545+0000","last_clean":"2024-09-11T05:22:27.598545+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:27.598545+0000","last_undegraded":"2024-09-11T05:22:27.598545+0000","last_fullsized":"2024-09-11T05:22:27.598545+0000","mapping_epoch":35,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18260,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18462","reported_seq":18599,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.947982+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:27.947982+0000","last_peered":"2024-09-11T05:22:27.947982+0000","last_clean":"2024-09-11T05:22:27.947982+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:27.947982+0000","last_undegraded":"2024-09-11T05:22:27.947982+0000","last_fullsized":"2024-09-11T05:22:27.947982+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15296,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18389","reported_seq":25744,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:31.146489+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:31.146489+0000","last_peered":"2024-09-11T05:22:31.146489+0000","last_clean":"2024-09-11T05:22:31.146489+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:31.146489+0000","last_undegraded":"2024-09-11T05:22:31.146489+0000","last_fullsized":"2024-09-11T05:22:31.146489+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6742016,"num_objects":1646,"num_object_clones":0,"num_object_copies":3292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14933,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18367","reported_seq":18993,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.915559+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:27.915559+0000","last_peered":"2024-09-11T05:22:27.915559+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:27.915559+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7516160,"num_objects":1835,"num_object_clones":0,"num_object_copies":3670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15532,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18142","reported_seq":18276,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.789446+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:27.789446+0000","last_peered":"2024-09-11T05:22:27.789446+0000","last_clean":"2024-09-11T05:22:27.789446+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:27.789446+0000","last_undegraded":"2024-09-11T05:22:27.789446+0000","last_fullsized":"2024-09-11T05:22:27.789446+0000","mapping_epoch":34,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7135232,"num_objects":1742,"num_object_clones":0,"num_object_copies":3484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18261","reported_seq":18403,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.543672+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:27.543672+0000","last_peered":"2024-09-11T05:22:27.543672+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:27.543672+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7004160,"num_objects":1710,"num_object_clones":0,"num_object_copies":3420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16329,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1815}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17965","reported_seq":18140,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.898516+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:27.898516+0000","last_peered":"2024-09-11T05:22:27.898516+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:27.898516+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6991872,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18262","reported_seq":18390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.557637+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:27.557637+0000","last_peered":"2024-09-11T05:22:27.557637+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:27.557637+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7254016,"num_objects":1771,"num_object_clones":0,"num_object_copies":3542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1751}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17981","reported_seq":25077,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:27.617392+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:27.617392+0000","last_peered":"2024-09-11T05:22:27.617392+0000","last_clean":"2024-09-11T05:22:27.617392+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:27.617392+0000","last_undegraded":"2024-09-11T05:22:27.617392+0000","last_fullsized":"2024-09-11T05:22:27.617392+0000","mapping_epoch":44,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7073792,"num_objects":1727,"num_object_clones":0,"num_object_copies":3454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15884,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18453","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.820496+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:27.820496+0000","last_peered":"2024-09-11T05:22:27.820496+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:27.820496+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15486,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17916","reported_seq":18060,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.865569+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:27.865569+0000","last_peered":"2024-09-11T05:22:27.865569+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:27.865569+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7118848,"num_objects":1738,"num_object_clones":0,"num_object_copies":3476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14782,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18103","reported_seq":18248,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.937085+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:27.937085+0000","last_peered":"2024-09-11T05:22:27.937085+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:27.937085+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17932","reported_seq":18086,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:27.896378+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:27.896378+0000","last_peered":"2024-09-11T05:22:27.896378+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:27.896378+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'7900","ondisk_log_start":"45'7900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6791785,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14939,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":112124545,"num_objects":27376,"num_object_clones":0,"num_object_copies":54752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254319,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10819,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":395141120,"data_stored":395070354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494648,"kb_used_data":170024,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93877192,"statfs":{"total":96636764160,"available":96130244608,"internally_reserved":0,"allocated":174104576,"data_stored":173824710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504676,"kb_used_data":80080,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93867164,"statfs":{"total":96636764160,"available":96119975936,"internally_reserved":0,"allocated":82001920,"data_stored":81665792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434761387},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572592,"kb_used_data":139736,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93799248,"statfs":{"total":96636764160,"available":96050429952,"internally_reserved":0,"allocated":143089664,"data_stored":142809793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":141893632,"data_stored":141890153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80412672,"data_stored":80352936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172834816,"data_stored":172827265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:35.702 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:36.442 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:36.445 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:36.477 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-11T05:22:35.628200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103903892,"num_objects":25371,"num_object_clones":0,"num_object_copies":50742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256328,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10889,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1573048,"kb_used_data":378488,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281542472,"statfs":{"total":289910292480,"available":288299491328,"internally_reserved":0,"allocated":387571712,"data_stored":386675847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1223150593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,5,4,7,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18493440,"num_objects":-4515,"num_object_clones":0,"num_object_copies":-9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005103"},"pg_stats":[{"pgid":"2.f","version":"49'18561","reported_seq":18650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.878354+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:32.878354+0000","last_peered":"2024-09-11T05:22:32.878354+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:32.878354+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672384,"num_objects":1629,"num_object_clones":0,"num_object_copies":3258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18561,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18372","reported_seq":18485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.583819+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:32.583819+0000","last_peered":"2024-09-11T05:22:32.583819+0000","last_clean":"2024-09-11T05:22:32.583819+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:32.583819+0000","last_undegraded":"2024-09-11T05:22:32.583819+0000","last_fullsized":"2024-09-11T05:22:32.583819+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6258688,"num_objects":1528,"num_object_clones":0,"num_object_copies":3056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16770,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18104","reported_seq":25241,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:31.085369+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:31.085369+0000","last_peered":"2024-09-11T05:22:31.085369+0000","last_clean":"2024-09-11T05:22:31.085369+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:31.085369+0000","last_undegraded":"2024-09-11T05:22:31.085369+0000","last_fullsized":"2024-09-11T05:22:31.085369+0000","mapping_epoch":48,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6606872,"num_objects":1614,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16507,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18389","reported_seq":18471,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.618086+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:32.618086+0000","last_peered":"2024-09-11T05:22:32.618086+0000","last_clean":"2024-09-11T05:22:32.618086+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:32.618086+0000","last_undegraded":"2024-09-11T05:22:32.618086+0000","last_fullsized":"2024-09-11T05:22:32.618086+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6434816,"num_objects":1571,"num_object_clones":0,"num_object_copies":3142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18389,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18611","reported_seq":18751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.916586+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:32.916586+0000","last_peered":"2024-09-11T05:22:32.916586+0000","last_clean":"2024-09-11T05:22:32.916586+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:32.916586+0000","last_undegraded":"2024-09-11T05:22:32.916586+0000","last_fullsized":"2024-09-11T05:22:32.916586+0000","mapping_epoch":34,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6381568,"num_objects":1558,"num_object_clones":0,"num_object_copies":3116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18389","reported_seq":25744,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:31.146489+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:31.146489+0000","last_peered":"2024-09-11T05:22:31.146489+0000","last_clean":"2024-09-11T05:22:31.146489+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:31.146489+0000","last_undegraded":"2024-09-11T05:22:31.146489+0000","last_fullsized":"2024-09-11T05:22:31.146489+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6742016,"num_objects":1646,"num_object_clones":0,"num_object_copies":3292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14933,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18507","reported_seq":19270,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.977599+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:32.977599+0000","last_peered":"2024-09-11T05:22:32.977599+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:32.977599+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15672,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18294","reported_seq":18428,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.926340+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:32.926340+0000","last_peered":"2024-09-11T05:22:32.926340+0000","last_clean":"2024-09-11T05:22:32.926340+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:32.926340+0000","last_undegraded":"2024-09-11T05:22:32.926340+0000","last_fullsized":"2024-09-11T05:22:32.926340+0000","mapping_epoch":34,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6512640,"num_objects":1590,"num_object_clones":0,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15597,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18415","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.605136+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:32.605136+0000","last_peered":"2024-09-11T05:22:32.605136+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:32.605136+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6373376,"num_objects":1556,"num_object_clones":0,"num_object_copies":3112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16483,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18106","reported_seq":18282,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.958618+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:32.958618+0000","last_peered":"2024-09-11T05:22:32.958618+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:32.958618+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6414336,"num_objects":1566,"num_object_clones":0,"num_object_copies":3132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14859,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1876}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18427","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.495004+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:32.495004+0000","last_peered":"2024-09-11T05:22:32.495004+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:32.495004+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6578176,"num_objects":1606,"num_object_clones":0,"num_object_copies":3212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16456,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1916}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18115","reported_seq":25211,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.476892+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:32.476892+0000","last_peered":"2024-09-11T05:22:32.476892+0000","last_clean":"2024-09-11T05:22:32.476892+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:32.476892+0000","last_undegraded":"2024-09-11T05:22:32.476892+0000","last_fullsized":"2024-09-11T05:22:32.476892+0000","mapping_epoch":44,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6524928,"num_objects":1593,"num_object_clones":0,"num_object_copies":3186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16018,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18608","reported_seq":18747,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.961883+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:32.961883+0000","last_peered":"2024-09-11T05:22:32.961883+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:32.961883+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18062","reported_seq":18206,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.874808+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:32.874808+0000","last_peered":"2024-09-11T05:22:32.874808+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:32.874808+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6520832,"num_objects":1592,"num_object_clones":0,"num_object_copies":3184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14928,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1903}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18225","reported_seq":18369,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.985583+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:32.985583+0000","last_peered":"2024-09-11T05:22:32.985583+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:32.985583+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18077","reported_seq":18231,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.929202+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:32.929202+0000","last_peered":"2024-09-11T05:22:32.929202+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:32.929202+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197865,"num_objects":1514,"num_object_clones":0,"num_object_copies":3028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":103903873,"num_objects":25369,"num_object_clones":0,"num_object_copies":50738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256326,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10888,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":383586304,"data_stored":383515538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160662,"ondisk_log_size":160662,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494160,"kb_used_data":165372,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93877680,"statfs":{"total":96636764160,"available":96130744320,"internally_reserved":0,"allocated":169340928,"data_stored":169061062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504676,"kb_used_data":80080,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93867164,"statfs":{"total":96636764160,"available":96119975936,"internally_reserved":0,"allocated":82001920,"data_stored":81665792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434761387},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215224,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574212,"kb_used_data":133036,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93797628,"statfs":{"total":96636764160,"available":96048771072,"internally_reserved":0,"allocated":136228864,"data_stored":135948993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":451735211},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":134938624,"data_stored":134935145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80412672,"data_stored":80352936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":168235008,"data_stored":168227457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:36.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:37.367 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:37.369 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:37.428 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-11T05:22:35.628200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":103903892,"num_objects":25371,"num_object_clones":0,"num_object_copies":50742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256328,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10889,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1573048,"kb_used_data":378488,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281542472,"statfs":{"total":289910292480,"available":288299491328,"internally_reserved":0,"allocated":387571712,"data_stored":386675847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1223150593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,5,4,7,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18493440,"num_objects":-4515,"num_object_clones":0,"num_object_copies":-9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005103"},"pg_stats":[{"pgid":"2.f","version":"49'18561","reported_seq":18650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.878354+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:32.878354+0000","last_peered":"2024-09-11T05:22:32.878354+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:32.878354+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672384,"num_objects":1629,"num_object_clones":0,"num_object_copies":3258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18561,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18372","reported_seq":18485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.583819+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:32.583819+0000","last_peered":"2024-09-11T05:22:32.583819+0000","last_clean":"2024-09-11T05:22:32.583819+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:32.583819+0000","last_undegraded":"2024-09-11T05:22:32.583819+0000","last_fullsized":"2024-09-11T05:22:32.583819+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6258688,"num_objects":1528,"num_object_clones":0,"num_object_copies":3056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16770,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18104","reported_seq":25241,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:31.085369+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:31.085369+0000","last_peered":"2024-09-11T05:22:31.085369+0000","last_clean":"2024-09-11T05:22:31.085369+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:31.085369+0000","last_undegraded":"2024-09-11T05:22:31.085369+0000","last_fullsized":"2024-09-11T05:22:31.085369+0000","mapping_epoch":48,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6606872,"num_objects":1614,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16507,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18389","reported_seq":18471,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.618086+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:32.618086+0000","last_peered":"2024-09-11T05:22:32.618086+0000","last_clean":"2024-09-11T05:22:32.618086+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:32.618086+0000","last_undegraded":"2024-09-11T05:22:32.618086+0000","last_fullsized":"2024-09-11T05:22:32.618086+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6434816,"num_objects":1571,"num_object_clones":0,"num_object_copies":3142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18389,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18611","reported_seq":18751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.916586+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:32.916586+0000","last_peered":"2024-09-11T05:22:32.916586+0000","last_clean":"2024-09-11T05:22:32.916586+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:32.916586+0000","last_undegraded":"2024-09-11T05:22:32.916586+0000","last_fullsized":"2024-09-11T05:22:32.916586+0000","mapping_epoch":34,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6381568,"num_objects":1558,"num_object_clones":0,"num_object_copies":3116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18389","reported_seq":25744,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:31.146489+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:31.146489+0000","last_peered":"2024-09-11T05:22:31.146489+0000","last_clean":"2024-09-11T05:22:31.146489+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:31.146489+0000","last_undegraded":"2024-09-11T05:22:31.146489+0000","last_fullsized":"2024-09-11T05:22:31.146489+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6742016,"num_objects":1646,"num_object_clones":0,"num_object_copies":3292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14933,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18507","reported_seq":19270,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.977599+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:32.977599+0000","last_peered":"2024-09-11T05:22:32.977599+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:32.977599+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15672,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18294","reported_seq":18428,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.926340+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:32.926340+0000","last_peered":"2024-09-11T05:22:32.926340+0000","last_clean":"2024-09-11T05:22:32.926340+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:32.926340+0000","last_undegraded":"2024-09-11T05:22:32.926340+0000","last_fullsized":"2024-09-11T05:22:32.926340+0000","mapping_epoch":34,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6512640,"num_objects":1590,"num_object_clones":0,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15597,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18415","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.605136+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:32.605136+0000","last_peered":"2024-09-11T05:22:32.605136+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:32.605136+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6373376,"num_objects":1556,"num_object_clones":0,"num_object_copies":3112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16483,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18106","reported_seq":18282,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.958618+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:32.958618+0000","last_peered":"2024-09-11T05:22:32.958618+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:32.958618+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6414336,"num_objects":1566,"num_object_clones":0,"num_object_copies":3132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14859,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1876}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18427","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.495004+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:32.495004+0000","last_peered":"2024-09-11T05:22:32.495004+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:32.495004+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6578176,"num_objects":1606,"num_object_clones":0,"num_object_copies":3212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16456,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1916}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750432+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750432+0000","last_peered":"2024-09-11T05:22:17.750432+0000","last_clean":"2024-09-11T05:22:17.750432+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750432+0000","last_undegraded":"2024-09-11T05:22:17.750432+0000","last_fullsized":"2024-09-11T05:22:17.750432+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18115","reported_seq":25211,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.476892+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:32.476892+0000","last_peered":"2024-09-11T05:22:32.476892+0000","last_clean":"2024-09-11T05:22:32.476892+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:32.476892+0000","last_undegraded":"2024-09-11T05:22:32.476892+0000","last_fullsized":"2024-09-11T05:22:32.476892+0000","mapping_epoch":44,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6524928,"num_objects":1593,"num_object_clones":0,"num_object_copies":3186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16018,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18608","reported_seq":18747,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.961883+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:32.961883+0000","last_peered":"2024-09-11T05:22:32.961883+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:32.961883+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750767+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750767+0000","last_peered":"2024-09-11T05:22:17.750767+0000","last_clean":"2024-09-11T05:22:17.750767+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750767+0000","last_undegraded":"2024-09-11T05:22:17.750767+0000","last_fullsized":"2024-09-11T05:22:17.750767+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18062","reported_seq":18206,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.874808+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:32.874808+0000","last_peered":"2024-09-11T05:22:32.874808+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:32.874808+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6520832,"num_objects":1592,"num_object_clones":0,"num_object_copies":3184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14928,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1903}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18225","reported_seq":18369,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.985583+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:32.985583+0000","last_peered":"2024-09-11T05:22:32.985583+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:32.985583+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750932+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750932+0000","last_peered":"2024-09-11T05:22:17.750932+0000","last_clean":"2024-09-11T05:22:17.750932+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750932+0000","last_undegraded":"2024-09-11T05:22:17.750932+0000","last_fullsized":"2024-09-11T05:22:17.750932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18077","reported_seq":18231,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.929202+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:32.929202+0000","last_peered":"2024-09-11T05:22:32.929202+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:32.929202+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197865,"num_objects":1514,"num_object_clones":0,"num_object_copies":3028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":103903873,"num_objects":25369,"num_object_clones":0,"num_object_copies":50738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256326,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10888,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":383586304,"data_stored":383515538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160662,"ondisk_log_size":160662,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494160,"kb_used_data":165372,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93877680,"statfs":{"total":96636764160,"available":96130744320,"internally_reserved":0,"allocated":169340928,"data_stored":169061062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504676,"kb_used_data":80080,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93867164,"statfs":{"total":96636764160,"available":96119975936,"internally_reserved":0,"allocated":82001920,"data_stored":81665792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434761387},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215224,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574212,"kb_used_data":133036,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93797628,"statfs":{"total":96636764160,"available":96048771072,"internally_reserved":0,"allocated":136228864,"data_stored":135948993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":451735211},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":134938624,"data_stored":134935145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80412672,"data_stored":80352936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":168235008,"data_stored":168227457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18561", 'reported_seq': 18650, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.878354+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:32.878354+0000', 'last_peered': '2024-09-11T05:22:32.878354+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:32.878354+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6672384, 'num_objects': 1629, 'num_object_clones': 0, 'num_object_copies': 3258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18561, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2046}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18507", 'reported_seq': 19270, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.977599+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:22:32.977599+0000', 'last_peered': '2024-09-11T05:22:32.977599+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:32.977599+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6942720, 'num_objects': 1695, 'num_object_clones': 0, 'num_object_copies': 3390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15672, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 310, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1573}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18415", 'reported_seq': 18555, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.605136+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:32.605136+0000', 'last_peered': '2024-09-11T05:22:32.605136+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:32.605136+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6373376, 'num_objects': 1556, 'num_object_clones': 0, 'num_object_copies': 3112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1556, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16483, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1969}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:37.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18106", 'reported_seq': 18282, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.958618+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:32.958618+0000', 'last_peered': '2024-09-11T05:22:32.958618+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:32.958618+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'8100", 'ondisk_log_start': "45'8100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6414336, 'num_objects': 1566, 'num_object_clones': 0, 'num_object_copies': 3132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14859, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1876}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18427", 'reported_seq': 18555, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.495004+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:32.495004+0000', 'last_peered': '2024-09-11T05:22:32.495004+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:32.495004+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6578176, 'num_objects': 1606, 'num_object_clones': 0, 'num_object_copies': 3212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16456, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1916}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18608", 'reported_seq': 18747, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.961883+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:32.961883+0000', 'last_peered': '2024-09-11T05:22:32.961883+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:32.961883+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6291456, 'num_objects': 1536, 'num_object_clones': 0, 'num_object_copies': 3072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1536, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15641, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2087}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:37.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18062", 'reported_seq': 18206, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.874808+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:32.874808+0000', 'last_peered': '2024-09-11T05:22:32.874808+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:32.874808+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'8000", 'ondisk_log_start': "45'8000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6520832, 'num_objects': 1592, 'num_object_clones': 0, 'num_object_copies': 3184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14928, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1903}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18225", 'reported_seq': 18369, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.985583+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:32.985583+0000', 'last_peered': '2024-09-11T05:22:32.985583+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:32.985583+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'8200", 'ondisk_log_start': "45'8200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6451200, 'num_objects': 1575, 'num_object_clones': 0, 'num_object_copies': 3150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14983, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1969}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:37.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18077", 'reported_seq': 18231, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:32.929202+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:32.929202+0000', 'last_peered': '2024-09-11T05:22:32.929202+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:32.929202+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'8000", 'ondisk_log_start': "45'8000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6197865, 'num_objects': 1514, 'num_object_clones': 0, 'num_object_copies': 3028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15084, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2001}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:37.432 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:38.364 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:38.364 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31624/50168 objects degraded (63.036%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":636,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":25084,"data_bytes":102728340,"bytes_used":1610903552,"bytes_avail":288299388928,"bytes_total":289910292480,"degraded_objects":31624,"degraded_total":50168,"degraded_ratio":0.63036198373465158,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":399},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:38.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:38.406 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:39.197 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:39.293 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-11T05:22:37.628661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":102728340,"num_objects":25084,"num_object_clones":0,"num_object_copies":50168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25084,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256615,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10889,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160652,"ondisk_log_size":160652,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1573148,"kb_used_data":370272,"kb_used_omap":12,"kb_used_meta":1202803,"kb_avail":281542372,"statfs":{"total":289910292480,"available":288299388928,"internally_reserved":0,"allocated":379158528,"data_stored":378262663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1231670273},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,6,5,11,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19668992,"num_objects":-4802,"num_object_clones":0,"num_object_copies":-9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005053"},"pg_stats":[{"pgid":"2.f","version":"49'18561","reported_seq":18650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.878354+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:32.878354+0000","last_peered":"2024-09-11T05:22:32.878354+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:32.878354+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672384,"num_objects":1629,"num_object_clones":0,"num_object_copies":3258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18561,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18372","reported_seq":18485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.583819+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:32.583819+0000","last_peered":"2024-09-11T05:22:32.583819+0000","last_clean":"2024-09-11T05:22:32.583819+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:32.583819+0000","last_undegraded":"2024-09-11T05:22:32.583819+0000","last_fullsized":"2024-09-11T05:22:32.583819+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6258688,"num_objects":1528,"num_object_clones":0,"num_object_copies":3056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16770,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18234","reported_seq":25371,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:36.192337+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:36.192337+0000","last_peered":"2024-09-11T05:22:36.192337+0000","last_clean":"2024-09-11T05:22:36.192337+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:36.192337+0000","last_undegraded":"2024-09-11T05:22:36.192337+0000","last_fullsized":"2024-09-11T05:22:36.192337+0000","mapping_epoch":48,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6074392,"num_objects":1484,"num_object_clones":0,"num_object_copies":2968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16637,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18389","reported_seq":18471,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.618086+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:32.618086+0000","last_peered":"2024-09-11T05:22:32.618086+0000","last_clean":"2024-09-11T05:22:32.618086+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:32.618086+0000","last_undegraded":"2024-09-11T05:22:32.618086+0000","last_fullsized":"2024-09-11T05:22:32.618086+0000","mapping_epoch":35,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6434816,"num_objects":1571,"num_object_clones":0,"num_object_copies":3142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18389,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18611","reported_seq":18751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.916586+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:32.916586+0000","last_peered":"2024-09-11T05:22:32.916586+0000","last_clean":"2024-09-11T05:22:32.916586+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:32.916586+0000","last_undegraded":"2024-09-11T05:22:32.916586+0000","last_fullsized":"2024-09-11T05:22:32.916586+0000","mapping_epoch":34,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6381568,"num_objects":1558,"num_object_clones":0,"num_object_copies":3116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18546","reported_seq":25902,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:36.179126+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:36.179126+0000","last_peered":"2024-09-11T05:22:36.179126+0000","last_clean":"2024-09-11T05:22:36.179126+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:36.179126+0000","last_undegraded":"2024-09-11T05:22:36.179126+0000","last_fullsized":"2024-09-11T05:22:36.179126+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6098944,"num_objects":1489,"num_object_clones":0,"num_object_copies":2978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15090,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18507","reported_seq":19270,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.977599+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:32.977599+0000","last_peered":"2024-09-11T05:22:32.977599+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:32.977599+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15672,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18294","reported_seq":18428,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.926340+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:32.926340+0000","last_peered":"2024-09-11T05:22:32.926340+0000","last_clean":"2024-09-11T05:22:32.926340+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:32.926340+0000","last_undegraded":"2024-09-11T05:22:32.926340+0000","last_fullsized":"2024-09-11T05:22:32.926340+0000","mapping_epoch":34,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6512640,"num_objects":1590,"num_object_clones":0,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15597,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18415","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.605136+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:32.605136+0000","last_peered":"2024-09-11T05:22:32.605136+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:32.605136+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6373376,"num_objects":1556,"num_object_clones":0,"num_object_copies":3112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16483,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18106","reported_seq":18282,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.958618+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:32.958618+0000","last_peered":"2024-09-11T05:22:32.958618+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:32.958618+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6414336,"num_objects":1566,"num_object_clones":0,"num_object_copies":3132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14859,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1876}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18427","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.495004+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:32.495004+0000","last_peered":"2024-09-11T05:22:32.495004+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:32.495004+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6578176,"num_objects":1606,"num_object_clones":0,"num_object_copies":3212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16456,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1916}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18115","reported_seq":25211,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:32.476892+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:32.476892+0000","last_peered":"2024-09-11T05:22:32.476892+0000","last_clean":"2024-09-11T05:22:32.476892+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:32.476892+0000","last_undegraded":"2024-09-11T05:22:32.476892+0000","last_fullsized":"2024-09-11T05:22:32.476892+0000","mapping_epoch":44,"log_start":"45'8100","ondisk_log_start":"45'8100","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6524928,"num_objects":1593,"num_object_clones":0,"num_object_copies":3186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16018,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18608","reported_seq":18747,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.961883+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:32.961883+0000","last_peered":"2024-09-11T05:22:32.961883+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:32.961883+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18062","reported_seq":18206,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.874808+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:32.874808+0000","last_peered":"2024-09-11T05:22:32.874808+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:32.874808+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6520832,"num_objects":1592,"num_object_clones":0,"num_object_copies":3184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14928,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1903}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18225","reported_seq":18369,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.985583+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:32.985583+0000","last_peered":"2024-09-11T05:22:32.985583+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:32.985583+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6451200,"num_objects":1575,"num_object_clones":0,"num_object_copies":3150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18077","reported_seq":18231,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:32.929202+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:32.929202+0000","last_peered":"2024-09-11T05:22:32.929202+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:32.929202+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8000","ondisk_log_start":"45'8000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197865,"num_objects":1514,"num_object_clones":0,"num_object_copies":3028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":102728321,"num_objects":25082,"num_object_clones":0,"num_object_copies":50164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25082,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256613,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10888,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376303616,"data_stored":376232850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160649,"ondisk_log_size":160649,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494160,"kb_used_data":165372,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93877680,"statfs":{"total":96636764160,"available":96130744320,"internally_reserved":0,"allocated":169340928,"data_stored":169061062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504776,"kb_used_data":71864,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93867064,"statfs":{"total":96636764160,"available":96119873536,"internally_reserved":0,"allocated":73588736,"data_stored":73252608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443281067},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,5,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215224,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574212,"kb_used_data":133036,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93797628,"statfs":{"total":96636764160,"available":96048771072,"internally_reserved":0,"allocated":136228864,"data_stored":135948993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":451735211},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":134938624,"data_stored":134935145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73129984,"data_stored":73070248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":168235008,"data_stored":168227457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:42.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:43.235 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:43.311 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-11T05:22:41.629575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":93344404,"num_objects":22793,"num_object_clones":0,"num_object_copies":45586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22793,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258906,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10965,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534048,"kb_used_data":351980,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281581472,"statfs":{"total":289910292480,"available":288339427328,"internally_reserved":0,"allocated":360427520,"data_stored":359531655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1210371073},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,2,3,6,10,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19935232,"num_objects":-4867,"num_object_clones":0,"num_object_copies":-9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4867,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005057"},"pg_stats":[{"pgid":"2.f","version":"49'18728","reported_seq":18815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:37.996656+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:37.996656+0000","last_peered":"2024-09-11T05:22:37.996656+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:37.996656+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5988352,"num_objects":1462,"num_object_clones":0,"num_object_copies":2924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18728,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2213}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18493","reported_seq":18606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:37.580933+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:37.580933+0000","last_peered":"2024-09-11T05:22:37.580933+0000","last_clean":"2024-09-11T05:22:37.580933+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:37.580933+0000","last_undegraded":"2024-09-11T05:22:37.580933+0000","last_fullsized":"2024-09-11T05:22:37.580933+0000","mapping_epoch":35,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5763072,"num_objects":1407,"num_object_clones":0,"num_object_copies":2814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16891,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18381","reported_seq":25517,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:41.321879+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:41.321879+0000","last_peered":"2024-09-11T05:22:41.321879+0000","last_clean":"2024-09-11T05:22:41.321879+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:41.321879+0000","last_undegraded":"2024-09-11T05:22:41.321879+0000","last_fullsized":"2024-09-11T05:22:41.321879+0000","mapping_epoch":48,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5472280,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16784,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18534","reported_seq":18615,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:37.673905+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:37.673905+0000","last_peered":"2024-09-11T05:22:37.673905+0000","last_clean":"2024-09-11T05:22:37.673905+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:37.673905+0000","last_undegraded":"2024-09-11T05:22:37.673905+0000","last_fullsized":"2024-09-11T05:22:37.673905+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5840896,"num_objects":1426,"num_object_clones":0,"num_object_copies":2852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18534,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18754","reported_seq":18894,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:37.976034+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:37.976034+0000","last_peered":"2024-09-11T05:22:37.976034+0000","last_clean":"2024-09-11T05:22:37.976034+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:37.976034+0000","last_undegraded":"2024-09-11T05:22:37.976034+0000","last_fullsized":"2024-09-11T05:22:37.976034+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5795840,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15588,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18678","reported_seq":26033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:41.279706+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:41.279706+0000","last_peered":"2024-09-11T05:22:41.279706+0000","last_clean":"2024-09-11T05:22:41.279706+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:41.279706+0000","last_undegraded":"2024-09-11T05:22:41.279706+0000","last_fullsized":"2024-09-11T05:22:41.279706+0000","mapping_epoch":46,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5558272,"num_objects":1357,"num_object_clones":0,"num_object_copies":2714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15222,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18663","reported_seq":19579,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:38.006853+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:38.006853+0000","last_peered":"2024-09-11T05:22:38.006853+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:38.006853+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6303744,"num_objects":1539,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1653}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18443","reported_seq":18577,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:38.016585+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:38.016585+0000","last_peered":"2024-09-11T05:22:38.016585+0000","last_clean":"2024-09-11T05:22:38.016585+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:38.016585+0000","last_undegraded":"2024-09-11T05:22:38.016585+0000","last_fullsized":"2024-09-11T05:22:38.016585+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5902336,"num_objects":1441,"num_object_clones":0,"num_object_copies":2882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15746,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18538","reported_seq":18678,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:37.470752+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:37.470752+0000","last_peered":"2024-09-11T05:22:37.470752+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:37.470752+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5869568,"num_objects":1433,"num_object_clones":0,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16606,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18257","reported_seq":18433,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:38.022342+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:38.022342+0000","last_peered":"2024-09-11T05:22:38.022342+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:38.022342+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5795840,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15010,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18560","reported_seq":18687,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:37.642654+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:37.642654+0000","last_peered":"2024-09-11T05:22:37.642654+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:37.642654+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6033408,"num_objects":1473,"num_object_clones":0,"num_object_copies":2946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16589,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18259","reported_seq":25355,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:37.660393+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:37.660393+0000","last_peered":"2024-09-11T05:22:37.660393+0000","last_clean":"2024-09-11T05:22:37.660393+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:37.660393+0000","last_undegraded":"2024-09-11T05:22:37.660393+0000","last_fullsized":"2024-09-11T05:22:37.660393+0000","mapping_epoch":44,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5935104,"num_objects":1449,"num_object_clones":0,"num_object_copies":2898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16162,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18742","reported_seq":18882,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:37.958821+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:37.958821+0000","last_peered":"2024-09-11T05:22:37.958821+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:37.958821+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5742592,"num_objects":1402,"num_object_clones":0,"num_object_copies":2804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15775,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18228","reported_seq":18371,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:38.024548+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:38.024548+0000","last_peered":"2024-09-11T05:22:38.024548+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:38.024548+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5840896,"num_objects":1426,"num_object_clones":0,"num_object_copies":2852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15094,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18364","reported_seq":18510,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:37.959266+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:37.959266+0000","last_peered":"2024-09-11T05:22:37.959266+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:37.959266+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5881856,"num_objects":1436,"num_object_clones":0,"num_object_copies":2872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2108}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18218","reported_seq":18372,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:37.985376+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:37.985376+0000","last_peered":"2024-09-11T05:22:37.985376+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:37.985376+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8200","ondisk_log_start":"45'8200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5620329,"num_objects":1373,"num_object_clones":0,"num_object_copies":2746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15225,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":93344385,"num_objects":22791,"num_object_clones":0,"num_object_copies":45582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22791,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258904,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10964,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":357502976,"data_stored":357432210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477404,"kb_used_data":161096,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93894436,"statfs":{"total":96636764160,"available":96147902464,"internally_reserved":0,"allocated":164962304,"data_stored":164682438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323874475},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502092,"kb_used_data":65020,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93869748,"statfs":{"total":96636764160,"available":96122621952,"internally_reserved":0,"allocated":66580480,"data_stored":66244352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447540907},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215225,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":125864,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":128884736,"data_stored":128604865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,1,6,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":127680512,"data_stored":127677033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66002944,"data_stored":65943208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163819520,"data_stored":163811969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:43.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:44.282 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:44.282 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:44.345 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-11T05:22:43.630025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":84935316,"num_objects":20740,"num_object_clones":0,"num_object_copies":41480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20740,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260959,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11035,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534976,"kb_used_data":340432,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281580544,"statfs":{"total":289910292480,"available":288338477056,"internally_reserved":0,"allocated":348602368,"data_stored":347706503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1223150593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,7,11,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27189248,"num_objects":-6638,"num_object_clones":0,"num_object_copies":-13276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6638,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005000"},"pg_stats":[{"pgid":"2.f","version":"49'18895","reported_seq":18984,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.115161+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:43.115161+0000","last_peered":"2024-09-11T05:22:43.115161+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:43.115161+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5304320,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18895,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18637","reported_seq":18750,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.573764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:42.573764+0000","last_peered":"2024-09-11T05:22:42.573764+0000","last_clean":"2024-09-11T05:22:42.573764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:42.573764+0000","last_undegraded":"2024-09-11T05:22:42.573764+0000","last_fullsized":"2024-09-11T05:22:42.573764+0000","mapping_epoch":35,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5173248,"num_objects":1263,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17035,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18381","reported_seq":25517,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:41.321879+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:41.321879+0000","last_peered":"2024-09-11T05:22:41.321879+0000","last_clean":"2024-09-11T05:22:41.321879+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:41.321879+0000","last_undegraded":"2024-09-11T05:22:41.321879+0000","last_fullsized":"2024-09-11T05:22:41.321879+0000","mapping_epoch":48,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5472280,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16784,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18676","reported_seq":18758,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.518913+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:42.518913+0000","last_peered":"2024-09-11T05:22:42.518913+0000","last_clean":"2024-09-11T05:22:42.518913+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:42.518913+0000","last_undegraded":"2024-09-11T05:22:42.518913+0000","last_fullsized":"2024-09-11T05:22:42.518913+0000","mapping_epoch":35,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5259264,"num_objects":1284,"num_object_clones":0,"num_object_copies":2568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18676,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18897","reported_seq":19037,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.978315+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:42.978315+0000","last_peered":"2024-09-11T05:22:42.978315+0000","last_clean":"2024-09-11T05:22:42.978315+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:42.978315+0000","last_undegraded":"2024-09-11T05:22:42.978315+0000","last_fullsized":"2024-09-11T05:22:42.978315+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210112,"num_objects":1272,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15731,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18678","reported_seq":26033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:41.279706+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:41.279706+0000","last_peered":"2024-09-11T05:22:41.279706+0000","last_clean":"2024-09-11T05:22:41.279706+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:41.279706+0000","last_undegraded":"2024-09-11T05:22:41.279706+0000","last_fullsized":"2024-09-11T05:22:41.279706+0000","mapping_epoch":46,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5558272,"num_objects":1357,"num_object_clones":0,"num_object_copies":2714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15222,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18844","reported_seq":19898,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.102131+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:43.102131+0000","last_peered":"2024-09-11T05:22:43.102131+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:43.102131+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562368,"num_objects":1358,"num_object_clones":0,"num_object_copies":2716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16009,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18586","reported_seq":18721,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:43.052449+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:43.052449+0000","last_peered":"2024-09-11T05:22:43.052449+0000","last_clean":"2024-09-11T05:22:43.052449+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:43.052449+0000","last_undegraded":"2024-09-11T05:22:43.052449+0000","last_fullsized":"2024-09-11T05:22:43.052449+0000","mapping_epoch":34,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5316608,"num_objects":1298,"num_object_clones":0,"num_object_copies":2596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18710","reported_seq":18851,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:42.653813+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:42.653813+0000","last_peered":"2024-09-11T05:22:42.653813+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:42.653813+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5165056,"num_objects":1261,"num_object_clones":0,"num_object_copies":2522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16778,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2264}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18381","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.102013+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:43.102013+0000","last_peered":"2024-09-11T05:22:43.102013+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:43.102013+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5287936,"num_objects":1291,"num_object_clones":0,"num_object_copies":2582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15134,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18684","reported_seq":18812,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:42.526909+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:42.526909+0000","last_peered":"2024-09-11T05:22:42.526909+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:42.526909+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525504,"num_objects":1349,"num_object_clones":0,"num_object_copies":2698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16713,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18411","reported_seq":25507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.645287+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:42.645287+0000","last_peered":"2024-09-11T05:22:42.645287+0000","last_clean":"2024-09-11T05:22:42.645287+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:42.645287+0000","last_undegraded":"2024-09-11T05:22:42.645287+0000","last_fullsized":"2024-09-11T05:22:42.645287+0000","mapping_epoch":44,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5312512,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16314,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18882","reported_seq":19024,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.045518+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:43.045518+0000","last_peered":"2024-09-11T05:22:43.045518+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:43.045518+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5169152,"num_objects":1262,"num_object_clones":0,"num_object_copies":2524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15915,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2361}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18358","reported_seq":18502,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.093845+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:43.093845+0000","last_peered":"2024-09-11T05:22:43.093845+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:43.093845+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15224,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18501","reported_seq":18647,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.106517+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:43.106517+0000","last_peered":"2024-09-11T05:22:43.106517+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:43.106517+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5320704,"num_objects":1299,"num_object_clones":0,"num_object_copies":2598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15259,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18372","reported_seq":18526,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:42.979234+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:42.979234+0000","last_peered":"2024-09-11T05:22:42.979234+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:42.979234+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4989545,"num_objects":1219,"num_object_clones":0,"num_object_copies":2438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15379,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":84935297,"num_objects":20738,"num_object_clones":0,"num_object_copies":41476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260957,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11034,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":345542656,"data_stored":345471890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476908,"kb_used_data":156444,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93894932,"statfs":{"total":96636764160,"available":96148410368,"internally_reserved":0,"allocated":160198656,"data_stored":159918790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502092,"kb_used_data":65020,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93869748,"statfs":{"total":96636764160,"available":96122621952,"internally_reserved":0,"allocated":66580480,"data_stored":66244352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447540907},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555976,"kb_used_data":118968,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93815864,"statfs":{"total":96636764160,"available":96067444736,"internally_reserved":0,"allocated":121823232,"data_stored":121543361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":120573952,"data_stored":120570473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66002944,"data_stored":65943208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158965760,"data_stored":158958209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18895", 'reported_seq': 18984, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:43.115161+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:43.115161+0000', 'last_peered': '2024-09-11T05:22:43.115161+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:43.115161+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5304320, 'num_objects': 1295, 'num_object_clones': 0, 'num_object_copies': 2590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18895, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2380}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18844", 'reported_seq': 19898, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:43.102131+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:22:43.102131+0000', 'last_peered': '2024-09-11T05:22:43.102131+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:43.102131+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5562368, 'num_objects': 1358, 'num_object_clones': 0, 'num_object_copies': 2716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16009, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 456, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18710", 'reported_seq': 18851, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:42.653813+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:42.653813+0000', 'last_peered': '2024-09-11T05:22:42.653813+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:42.653813+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5165056, 'num_objects': 1261, 'num_object_clones': 0, 'num_object_copies': 2522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16778, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2264}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:44.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18381", 'reported_seq': 18555, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:43.102013+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:43.102013+0000', 'last_peered': '2024-09-11T05:22:43.102013+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:43.102013+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5287936, 'num_objects': 1291, 'num_object_clones': 0, 'num_object_copies': 2582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15134, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2151}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18684", 'reported_seq': 18812, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:42.526909+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:42.526909+0000', 'last_peered': '2024-09-11T05:22:42.526909+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:42.526909+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5525504, 'num_objects': 1349, 'num_object_clones': 0, 'num_object_copies': 2698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1349, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16713, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2173}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18882", 'reported_seq': 19024, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:43.045518+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:43.045518+0000', 'last_peered': '2024-09-11T05:22:43.045518+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:43.045518+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5169152, 'num_objects': 1262, 'num_object_clones': 0, 'num_object_copies': 2524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15915, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2361}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18358", 'reported_seq': 18502, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:43.093845+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:43.093845+0000', 'last_peered': '2024-09-11T05:22:43.093845+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:43.093845+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5308416, 'num_objects': 1296, 'num_object_clones': 0, 'num_object_copies': 2592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15224, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2199}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18501", 'reported_seq': 18647, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:43.106517+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:43.106517+0000', 'last_peered': '2024-09-11T05:22:43.106517+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:43.106517+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5320704, 'num_objects': 1299, 'num_object_clones': 0, 'num_object_copies': 2598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15259, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:44.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18372", 'reported_seq': 18526, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:42.979234+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:42.979234+0000', 'last_peered': '2024-09-11T05:22:42.979234+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:42.979234+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4989545, 'num_objects': 1219, 'num_object_clones': 0, 'num_object_copies': 2438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1219, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15379, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:44.349 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:45.262 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:45.262 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31478/41480 objects degraded (75.887%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":643,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":20740,"data_bytes":84935316,"bytes_used":1571815424,"bytes_avail":288338477056,"bytes_total":289910292480,"degraded_objects":31478,"degraded_total":41480,"degraded_ratio":0.75887174541947933,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":552},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:45.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:45.352 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:46.122 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:46.123 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:46.177 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-11T05:22:45.630497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":84935316,"num_objects":20740,"num_object_clones":0,"num_object_copies":41480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20740,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260959,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11035,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534976,"kb_used_data":340432,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281580544,"statfs":{"total":289910292480,"available":288338477056,"internally_reserved":0,"allocated":348602368,"data_stored":347706503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1223150593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,7,11,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18968576,"num_objects":-4631,"num_object_clones":0,"num_object_copies":-9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4631,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005032"},"pg_stats":[{"pgid":"2.f","version":"49'18895","reported_seq":18984,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.115161+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:43.115161+0000","last_peered":"2024-09-11T05:22:43.115161+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:43.115161+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5304320,"num_objects":1295,"num_object_clones":0,"num_object_copies":2590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18895,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18637","reported_seq":18750,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.573764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:42.573764+0000","last_peered":"2024-09-11T05:22:42.573764+0000","last_clean":"2024-09-11T05:22:42.573764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:42.573764+0000","last_undegraded":"2024-09-11T05:22:42.573764+0000","last_fullsized":"2024-09-11T05:22:42.573764+0000","mapping_epoch":35,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5173248,"num_objects":1263,"num_object_clones":0,"num_object_copies":2526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17035,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18381","reported_seq":25517,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:41.321879+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:41.321879+0000","last_peered":"2024-09-11T05:22:41.321879+0000","last_clean":"2024-09-11T05:22:41.321879+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:41.321879+0000","last_undegraded":"2024-09-11T05:22:41.321879+0000","last_fullsized":"2024-09-11T05:22:41.321879+0000","mapping_epoch":48,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5472280,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16784,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18676","reported_seq":18758,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.518913+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:42.518913+0000","last_peered":"2024-09-11T05:22:42.518913+0000","last_clean":"2024-09-11T05:22:42.518913+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:42.518913+0000","last_undegraded":"2024-09-11T05:22:42.518913+0000","last_fullsized":"2024-09-11T05:22:42.518913+0000","mapping_epoch":35,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5259264,"num_objects":1284,"num_object_clones":0,"num_object_copies":2568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18676,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18897","reported_seq":19037,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.978315+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:42.978315+0000","last_peered":"2024-09-11T05:22:42.978315+0000","last_clean":"2024-09-11T05:22:42.978315+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:42.978315+0000","last_undegraded":"2024-09-11T05:22:42.978315+0000","last_fullsized":"2024-09-11T05:22:42.978315+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5210112,"num_objects":1272,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15731,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18678","reported_seq":26033,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:41.279706+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:41.279706+0000","last_peered":"2024-09-11T05:22:41.279706+0000","last_clean":"2024-09-11T05:22:41.279706+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:41.279706+0000","last_undegraded":"2024-09-11T05:22:41.279706+0000","last_fullsized":"2024-09-11T05:22:41.279706+0000","mapping_epoch":46,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5558272,"num_objects":1357,"num_object_clones":0,"num_object_copies":2714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15222,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18844","reported_seq":19898,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.102131+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:43.102131+0000","last_peered":"2024-09-11T05:22:43.102131+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:43.102131+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562368,"num_objects":1358,"num_object_clones":0,"num_object_copies":2716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16009,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18586","reported_seq":18721,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:43.052449+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:43.052449+0000","last_peered":"2024-09-11T05:22:43.052449+0000","last_clean":"2024-09-11T05:22:43.052449+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:43.052449+0000","last_undegraded":"2024-09-11T05:22:43.052449+0000","last_fullsized":"2024-09-11T05:22:43.052449+0000","mapping_epoch":34,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5316608,"num_objects":1298,"num_object_clones":0,"num_object_copies":2596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18710","reported_seq":18851,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:42.653813+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:42.653813+0000","last_peered":"2024-09-11T05:22:42.653813+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:42.653813+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5165056,"num_objects":1261,"num_object_clones":0,"num_object_copies":2522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16778,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2264}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18381","reported_seq":18555,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.102013+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:43.102013+0000","last_peered":"2024-09-11T05:22:43.102013+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:43.102013+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5287936,"num_objects":1291,"num_object_clones":0,"num_object_copies":2582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15134,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18684","reported_seq":18812,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:42.526909+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:42.526909+0000","last_peered":"2024-09-11T05:22:42.526909+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:42.526909+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5525504,"num_objects":1349,"num_object_clones":0,"num_object_copies":2698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16713,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2173}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18411","reported_seq":25507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:42.645287+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:42.645287+0000","last_peered":"2024-09-11T05:22:42.645287+0000","last_clean":"2024-09-11T05:22:42.645287+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:42.645287+0000","last_undegraded":"2024-09-11T05:22:42.645287+0000","last_fullsized":"2024-09-11T05:22:42.645287+0000","mapping_epoch":44,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5312512,"num_objects":1297,"num_object_clones":0,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16314,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18882","reported_seq":19024,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.045518+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:43.045518+0000","last_peered":"2024-09-11T05:22:43.045518+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:43.045518+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5169152,"num_objects":1262,"num_object_clones":0,"num_object_copies":2524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15915,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2361}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18358","reported_seq":18502,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.093845+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:43.093845+0000","last_peered":"2024-09-11T05:22:43.093845+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:43.093845+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15224,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18501","reported_seq":18647,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:43.106517+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:43.106517+0000","last_peered":"2024-09-11T05:22:43.106517+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:43.106517+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5320704,"num_objects":1299,"num_object_clones":0,"num_object_copies":2598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15259,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18372","reported_seq":18526,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:42.979234+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:42.979234+0000","last_peered":"2024-09-11T05:22:42.979234+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:42.979234+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4989545,"num_objects":1219,"num_object_clones":0,"num_object_copies":2438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15379,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":84935297,"num_objects":20738,"num_object_clones":0,"num_object_copies":41476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260957,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11034,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":345542656,"data_stored":345471890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476908,"kb_used_data":156444,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93894932,"statfs":{"total":96636764160,"available":96148410368,"internally_reserved":0,"allocated":160198656,"data_stored":159918790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502092,"kb_used_data":65020,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93869748,"statfs":{"total":96636764160,"available":96122621952,"internally_reserved":0,"allocated":66580480,"data_stored":66244352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447540907},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555976,"kb_used_data":118968,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93815864,"statfs":{"total":96636764160,"available":96067444736,"internally_reserved":0,"allocated":121823232,"data_stored":121543361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":120573952,"data_stored":120570473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66002944,"data_stored":65943208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158965760,"data_stored":158958209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:49.179 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:50.086 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:50.147 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-11T05:22:49.631383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75670164,"num_objects":18478,"num_object_clones":0,"num_object_copies":36956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263221,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11105,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518216,"kb_used_data":322260,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281597304,"statfs":{"total":289910292480,"available":288355639296,"internally_reserved":0,"allocated":329994240,"data_stored":329098375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1224592385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,3,7,10,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27058176,"num_objects":-6606,"num_object_clones":0,"num_object_copies":-13212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6606,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002722"},"pg_stats":[{"pgid":"2.f","version":"49'19043","reported_seq":19132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.151817+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:48.151817+0000","last_peered":"2024-09-11T05:22:48.151817+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:48.151817+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4698112,"num_objects":1147,"num_object_clones":0,"num_object_copies":2294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19043,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18798","reported_seq":18911,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.665214+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:47.665214+0000","last_peered":"2024-09-11T05:22:47.665214+0000","last_clean":"2024-09-11T05:22:47.665214+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:47.665214+0000","last_undegraded":"2024-09-11T05:22:47.665214+0000","last_fullsized":"2024-09-11T05:22:47.665214+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4513792,"num_objects":1102,"num_object_clones":0,"num_object_copies":2204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17196,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18526","reported_seq":25663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:46.323500+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:46.323500+0000","last_peered":"2024-09-11T05:22:46.323500+0000","last_clean":"2024-09-11T05:22:46.323500+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:46.323500+0000","last_undegraded":"2024-09-11T05:22:46.323500+0000","last_fullsized":"2024-09-11T05:22:46.323500+0000","mapping_epoch":48,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4878360,"num_objects":1192,"num_object_clones":0,"num_object_copies":2384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1192,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16929,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18799","reported_seq":18881,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.687414+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:47.687414+0000","last_peered":"2024-09-11T05:22:47.687414+0000","last_clean":"2024-09-11T05:22:47.687414+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:47.687414+0000","last_undegraded":"2024-09-11T05:22:47.687414+0000","last_fullsized":"2024-09-11T05:22:47.687414+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4755456,"num_objects":1161,"num_object_clones":0,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18799,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19025","reported_seq":19165,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:48.108858+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:48.108858+0000","last_peered":"2024-09-11T05:22:48.108858+0000","last_clean":"2024-09-11T05:22:48.108858+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:48.108858+0000","last_undegraded":"2024-09-11T05:22:48.108858+0000","last_fullsized":"2024-09-11T05:22:48.108858+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4685824,"num_objects":1144,"num_object_clones":0,"num_object_copies":2288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15859,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18808","reported_seq":26164,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:46.327937+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:46.327937+0000","last_peered":"2024-09-11T05:22:46.327937+0000","last_clean":"2024-09-11T05:22:46.327937+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:46.327937+0000","last_undegraded":"2024-09-11T05:22:46.327937+0000","last_fullsized":"2024-09-11T05:22:46.327937+0000","mapping_epoch":46,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5025792,"num_objects":1227,"num_object_clones":0,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15352,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18985","reported_seq":20181,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.112771+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:48.112771+0000","last_peered":"2024-09-11T05:22:48.112771+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:48.112771+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4984832,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16150,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18750","reported_seq":18885,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:48.149550+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:48.149550+0000","last_peered":"2024-09-11T05:22:48.149550+0000","last_clean":"2024-09-11T05:22:48.149550+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:48.149550+0000","last_undegraded":"2024-09-11T05:22:48.149550+0000","last_fullsized":"2024-09-11T05:22:48.149550+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4644864,"num_objects":1134,"num_object_clones":0,"num_object_copies":2268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16053,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18857","reported_seq":18999,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:47.654167+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:47.654167+0000","last_peered":"2024-09-11T05:22:47.654167+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:47.654167+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4562944,"num_objects":1114,"num_object_clones":0,"num_object_copies":2228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16925,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18520","reported_seq":18696,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.043334+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:48.043334+0000","last_peered":"2024-09-11T05:22:48.043334+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:48.043334+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":1152,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15273,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18840","reported_seq":18968,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:47.658545+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:47.658545+0000","last_peered":"2024-09-11T05:22:47.658545+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:47.658545+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4886528,"num_objects":1193,"num_object_clones":0,"num_object_copies":2386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16869,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18541","reported_seq":25637,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.579190+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:47.579190+0000","last_peered":"2024-09-11T05:22:47.579190+0000","last_clean":"2024-09-11T05:22:47.579190+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:47.579190+0000","last_undegraded":"2024-09-11T05:22:47.579190+0000","last_fullsized":"2024-09-11T05:22:47.579190+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16444,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19019","reported_seq":19161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.081007+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:48.081007+0000","last_peered":"2024-09-11T05:22:48.081007+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:48.081007+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4608000,"num_objects":1125,"num_object_clones":0,"num_object_copies":2250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16052,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2498}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18508","reported_seq":18652,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.082314+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:48.082314+0000","last_peered":"2024-09-11T05:22:48.082314+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:48.082314+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4694016,"num_objects":1146,"num_object_clones":0,"num_object_copies":2292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15374,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18642","reported_seq":18788,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.114965+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:48.114965+0000","last_peered":"2024-09-11T05:22:48.114965+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:48.114965+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4743168,"num_objects":1158,"num_object_clones":0,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15400,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18494","reported_seq":18648,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.080212+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:48.080212+0000","last_peered":"2024-09-11T05:22:48.080212+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:48.080212+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489833,"num_objects":1097,"num_object_clones":0,"num_object_copies":2194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15501,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":75670145,"num_objects":18476,"num_object_clones":0,"num_object_copies":36952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263219,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11104,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":326848512,"data_stored":326777746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476484,"kb_used_data":151860,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93895356,"statfs":{"total":96636764160,"available":96148844544,"internally_reserved":0,"allocated":155504640,"data_stored":155224774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505028,"kb_used_data":58224,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93866812,"statfs":{"total":96636764160,"available":96119615488,"internally_reserved":0,"allocated":59621376,"data_stored":59285248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215227,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536704,"kb_used_data":112176,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93835136,"statfs":{"total":96636764160,"available":96087179264,"internally_reserved":0,"allocated":114868224,"data_stored":114588353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434695851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,9,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113577984,"data_stored":113574505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59035648,"data_stored":58975912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154234880,"data_stored":154227329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:50.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:50.938 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:51.034 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-11T05:22:49.631383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75670164,"num_objects":18478,"num_object_clones":0,"num_object_copies":36956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263221,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11105,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518216,"kb_used_data":322260,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281597304,"statfs":{"total":289910292480,"available":288355639296,"internally_reserved":0,"allocated":329994240,"data_stored":329098375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1224592385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,3,7,10,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27058176,"num_objects":-6606,"num_object_clones":0,"num_object_copies":-13212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6606,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002722"},"pg_stats":[{"pgid":"2.f","version":"49'19043","reported_seq":19132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.151817+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:48.151817+0000","last_peered":"2024-09-11T05:22:48.151817+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:48.151817+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4698112,"num_objects":1147,"num_object_clones":0,"num_object_copies":2294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19043,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18798","reported_seq":18911,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.665214+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:47.665214+0000","last_peered":"2024-09-11T05:22:47.665214+0000","last_clean":"2024-09-11T05:22:47.665214+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:47.665214+0000","last_undegraded":"2024-09-11T05:22:47.665214+0000","last_fullsized":"2024-09-11T05:22:47.665214+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4513792,"num_objects":1102,"num_object_clones":0,"num_object_copies":2204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17196,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18526","reported_seq":25663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:46.323500+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:46.323500+0000","last_peered":"2024-09-11T05:22:46.323500+0000","last_clean":"2024-09-11T05:22:46.323500+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:46.323500+0000","last_undegraded":"2024-09-11T05:22:46.323500+0000","last_fullsized":"2024-09-11T05:22:46.323500+0000","mapping_epoch":48,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4878360,"num_objects":1192,"num_object_clones":0,"num_object_copies":2384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1192,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16929,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18799","reported_seq":18881,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.687414+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:47.687414+0000","last_peered":"2024-09-11T05:22:47.687414+0000","last_clean":"2024-09-11T05:22:47.687414+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:47.687414+0000","last_undegraded":"2024-09-11T05:22:47.687414+0000","last_fullsized":"2024-09-11T05:22:47.687414+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4755456,"num_objects":1161,"num_object_clones":0,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18799,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19025","reported_seq":19165,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:48.108858+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:48.108858+0000","last_peered":"2024-09-11T05:22:48.108858+0000","last_clean":"2024-09-11T05:22:48.108858+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:48.108858+0000","last_undegraded":"2024-09-11T05:22:48.108858+0000","last_fullsized":"2024-09-11T05:22:48.108858+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4685824,"num_objects":1144,"num_object_clones":0,"num_object_copies":2288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15859,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18808","reported_seq":26164,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:46.327937+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:46.327937+0000","last_peered":"2024-09-11T05:22:46.327937+0000","last_clean":"2024-09-11T05:22:46.327937+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:46.327937+0000","last_undegraded":"2024-09-11T05:22:46.327937+0000","last_fullsized":"2024-09-11T05:22:46.327937+0000","mapping_epoch":46,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5025792,"num_objects":1227,"num_object_clones":0,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15352,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18985","reported_seq":20181,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.112771+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:48.112771+0000","last_peered":"2024-09-11T05:22:48.112771+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:48.112771+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4984832,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16150,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18750","reported_seq":18885,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:48.149550+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:48.149550+0000","last_peered":"2024-09-11T05:22:48.149550+0000","last_clean":"2024-09-11T05:22:48.149550+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:48.149550+0000","last_undegraded":"2024-09-11T05:22:48.149550+0000","last_fullsized":"2024-09-11T05:22:48.149550+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4644864,"num_objects":1134,"num_object_clones":0,"num_object_copies":2268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16053,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18857","reported_seq":18999,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:47.654167+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:47.654167+0000","last_peered":"2024-09-11T05:22:47.654167+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:47.654167+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4562944,"num_objects":1114,"num_object_clones":0,"num_object_copies":2228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16925,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18520","reported_seq":18696,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.043334+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:48.043334+0000","last_peered":"2024-09-11T05:22:48.043334+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:48.043334+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":1152,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15273,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18840","reported_seq":18968,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:47.658545+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:47.658545+0000","last_peered":"2024-09-11T05:22:47.658545+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:47.658545+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4886528,"num_objects":1193,"num_object_clones":0,"num_object_copies":2386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16869,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18541","reported_seq":25637,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.579190+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:47.579190+0000","last_peered":"2024-09-11T05:22:47.579190+0000","last_clean":"2024-09-11T05:22:47.579190+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:47.579190+0000","last_undegraded":"2024-09-11T05:22:47.579190+0000","last_fullsized":"2024-09-11T05:22:47.579190+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16444,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19019","reported_seq":19161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.081007+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:48.081007+0000","last_peered":"2024-09-11T05:22:48.081007+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:48.081007+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4608000,"num_objects":1125,"num_object_clones":0,"num_object_copies":2250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16052,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2498}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18508","reported_seq":18652,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.082314+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:48.082314+0000","last_peered":"2024-09-11T05:22:48.082314+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:48.082314+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4694016,"num_objects":1146,"num_object_clones":0,"num_object_copies":2292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15374,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18642","reported_seq":18788,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.114965+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:48.114965+0000","last_peered":"2024-09-11T05:22:48.114965+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:48.114965+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4743168,"num_objects":1158,"num_object_clones":0,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15400,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18494","reported_seq":18648,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.080212+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:48.080212+0000","last_peered":"2024-09-11T05:22:48.080212+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:48.080212+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489833,"num_objects":1097,"num_object_clones":0,"num_object_copies":2194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15501,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":75670145,"num_objects":18476,"num_object_clones":0,"num_object_copies":36952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263219,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11104,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":326848512,"data_stored":326777746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476484,"kb_used_data":151860,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93895356,"statfs":{"total":96636764160,"available":96148844544,"internally_reserved":0,"allocated":155504640,"data_stored":155224774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505028,"kb_used_data":58224,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93866812,"statfs":{"total":96636764160,"available":96119615488,"internally_reserved":0,"allocated":59621376,"data_stored":59285248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215227,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536704,"kb_used_data":112176,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93835136,"statfs":{"total":96636764160,"available":96087179264,"internally_reserved":0,"allocated":114868224,"data_stored":114588353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434695851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,9,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113577984,"data_stored":113574505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59035648,"data_stored":58975912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154234880,"data_stored":154227329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:51.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:51.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19043", 'reported_seq': 19132, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:48.151817+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:48.151817+0000', 'last_peered': '2024-09-11T05:22:48.151817+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:48.151817+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4698112, 'num_objects': 1147, 'num_object_clones': 0, 'num_object_copies': 2294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19043, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2528}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:51.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:51.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18985", 'reported_seq': 20181, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:48.112771+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:22:48.112771+0000', 'last_peered': '2024-09-11T05:22:48.112771+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:48.112771+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4984832, 'num_objects': 1217, 'num_object_clones': 0, 'num_object_copies': 2434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3052, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16150, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 526, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:51.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:51.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'18857", 'reported_seq': 18999, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:47.654167+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:47.654167+0000', 'last_peered': '2024-09-11T05:22:47.654167+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:47.654167+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4562944, 'num_objects': 1114, 'num_object_clones': 0, 'num_object_copies': 2228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1114, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16925, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2411}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:51.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:51.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18520", 'reported_seq': 18696, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:48.043334+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:48.043334+0000', 'last_peered': '2024-09-11T05:22:48.043334+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:48.043334+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4718592, 'num_objects': 1152, 'num_object_clones': 0, 'num_object_copies': 2304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1152, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15273, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2290}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:51.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:51.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18840", 'reported_seq': 18968, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:47.658545+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:47.658545+0000', 'last_peered': '2024-09-11T05:22:47.658545+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:47.658545+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4886528, 'num_objects': 1193, 'num_object_clones': 0, 'num_object_copies': 2386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1193, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16869, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2329}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:51.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:51.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19019", 'reported_seq': 19161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:48.081007+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:48.081007+0000', 'last_peered': '2024-09-11T05:22:48.081007+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:48.081007+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4608000, 'num_objects': 1125, 'num_object_clones': 0, 'num_object_copies': 2250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16052, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2498}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:51.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:51.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18508", 'reported_seq': 18652, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:48.082314+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:48.082314+0000', 'last_peered': '2024-09-11T05:22:48.082314+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:48.082314+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4694016, 'num_objects': 1146, 'num_object_clones': 0, 'num_object_copies': 2292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15374, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2349}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:51.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:51.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18642", 'reported_seq': 18788, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:48.114965+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:48.114965+0000', 'last_peered': '2024-09-11T05:22:48.114965+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:48.114965+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4743168, 'num_objects': 1158, 'num_object_clones': 0, 'num_object_copies': 2316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15400, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2386}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:51.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:51.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18494", 'reported_seq': 18648, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:48.080212+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:48.080212+0000', 'last_peered': '2024-09-11T05:22:48.080212+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:48.080212+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4489833, 'num_objects': 1097, 'num_object_clones': 0, 'num_object_copies': 2194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15501, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2418}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:51.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:51.974 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:51.975 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31408/36394 objects degraded (86.300%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":649,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":18197,"data_bytes":74519188,"bytes_used":1552019456,"bytes_avail":288358273024,"bytes_total":289910292480,"degraded_objects":31408,"degraded_total":36394,"degraded_ratio":0.86299939550475357,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":406},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:52.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:52.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:52.724 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:52.731 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:52.772 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-11T05:22:51.631748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":74519188,"num_objects":18197,"num_object_clones":0,"num_object_copies":36394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263502,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11105,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515644,"kb_used_data":315532,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281599876,"statfs":{"total":289910292480,"available":288358273024,"internally_reserved":0,"allocated":323104768,"data_stored":322208903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1228852225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,3,9,14,6,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19968000,"num_objects":-4875,"num_object_clones":0,"num_object_copies":-9750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4875,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002642"},"pg_stats":[{"pgid":"2.f","version":"49'19043","reported_seq":19132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.151817+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:48.151817+0000","last_peered":"2024-09-11T05:22:48.151817+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:48.151817+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4698112,"num_objects":1147,"num_object_clones":0,"num_object_copies":2294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19043,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18798","reported_seq":18911,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.665214+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:47.665214+0000","last_peered":"2024-09-11T05:22:47.665214+0000","last_clean":"2024-09-11T05:22:47.665214+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:47.665214+0000","last_undegraded":"2024-09-11T05:22:47.665214+0000","last_fullsized":"2024-09-11T05:22:47.665214+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4513792,"num_objects":1102,"num_object_clones":0,"num_object_copies":2204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17196,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18670","reported_seq":25807,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:51.390427+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:51.390427+0000","last_peered":"2024-09-11T05:22:51.390427+0000","last_clean":"2024-09-11T05:22:51.390427+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:51.390427+0000","last_undegraded":"2024-09-11T05:22:51.390427+0000","last_fullsized":"2024-09-11T05:22:51.390427+0000","mapping_epoch":48,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4288536,"num_objects":1048,"num_object_clones":0,"num_object_copies":2096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17073,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18799","reported_seq":18881,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.687414+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:47.687414+0000","last_peered":"2024-09-11T05:22:47.687414+0000","last_clean":"2024-09-11T05:22:47.687414+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:47.687414+0000","last_undegraded":"2024-09-11T05:22:47.687414+0000","last_fullsized":"2024-09-11T05:22:47.687414+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4755456,"num_objects":1161,"num_object_clones":0,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18799,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19025","reported_seq":19165,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:48.108858+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:48.108858+0000","last_peered":"2024-09-11T05:22:48.108858+0000","last_clean":"2024-09-11T05:22:48.108858+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:48.108858+0000","last_undegraded":"2024-09-11T05:22:48.108858+0000","last_fullsized":"2024-09-11T05:22:48.108858+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4685824,"num_objects":1144,"num_object_clones":0,"num_object_copies":2288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15859,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18945","reported_seq":26301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:51.438775+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:51.438775+0000","last_peered":"2024-09-11T05:22:51.438775+0000","last_clean":"2024-09-11T05:22:51.438775+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:51.438775+0000","last_undegraded":"2024-09-11T05:22:51.438775+0000","last_fullsized":"2024-09-11T05:22:51.438775+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464640,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15489,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'18985","reported_seq":20181,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.112771+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:48.112771+0000","last_peered":"2024-09-11T05:22:48.112771+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:48.112771+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4984832,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16150,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18750","reported_seq":18885,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:48.149550+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:48.149550+0000","last_peered":"2024-09-11T05:22:48.149550+0000","last_clean":"2024-09-11T05:22:48.149550+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:48.149550+0000","last_undegraded":"2024-09-11T05:22:48.149550+0000","last_fullsized":"2024-09-11T05:22:48.149550+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4644864,"num_objects":1134,"num_object_clones":0,"num_object_copies":2268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16053,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18857","reported_seq":18999,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:47.654167+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:47.654167+0000","last_peered":"2024-09-11T05:22:47.654167+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:47.654167+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4562944,"num_objects":1114,"num_object_clones":0,"num_object_copies":2228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16925,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18520","reported_seq":18696,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.043334+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:48.043334+0000","last_peered":"2024-09-11T05:22:48.043334+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:48.043334+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718592,"num_objects":1152,"num_object_clones":0,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15273,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18840","reported_seq":18968,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:47.658545+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:47.658545+0000","last_peered":"2024-09-11T05:22:47.658545+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:47.658545+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4886528,"num_objects":1193,"num_object_clones":0,"num_object_copies":2386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16869,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18541","reported_seq":25637,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:47.579190+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:47.579190+0000","last_peered":"2024-09-11T05:22:47.579190+0000","last_clean":"2024-09-11T05:22:47.579190+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:47.579190+0000","last_undegraded":"2024-09-11T05:22:47.579190+0000","last_fullsized":"2024-09-11T05:22:47.579190+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16444,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19019","reported_seq":19161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.081007+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:48.081007+0000","last_peered":"2024-09-11T05:22:48.081007+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:48.081007+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4608000,"num_objects":1125,"num_object_clones":0,"num_object_copies":2250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16052,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2498}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18508","reported_seq":18652,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.082314+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:48.082314+0000","last_peered":"2024-09-11T05:22:48.082314+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:48.082314+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4694016,"num_objects":1146,"num_object_clones":0,"num_object_copies":2292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15374,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18642","reported_seq":18788,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.114965+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:48.114965+0000","last_peered":"2024-09-11T05:22:48.114965+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:48.114965+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4743168,"num_objects":1158,"num_object_clones":0,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15400,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18494","reported_seq":18648,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:48.080212+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:48.080212+0000","last_peered":"2024-09-11T05:22:48.080212+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:48.080212+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489833,"num_objects":1097,"num_object_clones":0,"num_object_copies":2194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15501,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":74519169,"num_objects":18195,"num_object_clones":0,"num_object_copies":36390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263500,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11104,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319791104,"data_stored":319720338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476484,"kb_used_data":151860,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93895356,"statfs":{"total":96636764160,"available":96148844544,"internally_reserved":0,"allocated":155504640,"data_stored":155224774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502456,"kb_used_data":51496,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93869384,"statfs":{"total":96636764160,"available":96122249216,"internally_reserved":0,"allocated":52731904,"data_stored":52395776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3,3,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215227,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536704,"kb_used_data":112176,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93835136,"statfs":{"total":96636764160,"available":96087179264,"internally_reserved":0,"allocated":114868224,"data_stored":114588353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434695851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,9,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113577984,"data_stored":113574505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51978240,"data_stored":51918504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154234880,"data_stored":154227329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:55.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:56.507 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:56.507 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:56.583 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-11T05:22:55.632667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66429588,"num_objects":16222,"num_object_clones":0,"num_object_copies":32444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265477,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11178,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512688,"kb_used_data":304248,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281602832,"statfs":{"total":289910292480,"available":288361299968,"internally_reserved":0,"allocated":311549952,"data_stored":310654087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1237371905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,8,5,10,7,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18505728,"num_objects":-4518,"num_object_clones":0,"num_object_copies":-9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4518,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002642"},"pg_stats":[{"pgid":"2.f","version":"49'19172","reported_seq":19261,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.198543+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:53.198543+0000","last_peered":"2024-09-11T05:22:53.198543+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:53.198543+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4169728,"num_objects":1018,"num_object_clones":0,"num_object_copies":2036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19172,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2657}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18952","reported_seq":19064,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.775534+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:52.775534+0000","last_peered":"2024-09-11T05:22:52.775534+0000","last_clean":"2024-09-11T05:22:52.775534+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:52.775534+0000","last_undegraded":"2024-09-11T05:22:52.775534+0000","last_fullsized":"2024-09-11T05:22:52.775534+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17350,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18670","reported_seq":25807,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:51.390427+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:51.390427+0000","last_peered":"2024-09-11T05:22:51.390427+0000","last_clean":"2024-09-11T05:22:51.390427+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:51.390427+0000","last_undegraded":"2024-09-11T05:22:51.390427+0000","last_fullsized":"2024-09-11T05:22:51.390427+0000","mapping_epoch":48,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4288536,"num_objects":1048,"num_object_clones":0,"num_object_copies":2096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17073,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18918","reported_seq":19000,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.726654+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:52.726654+0000","last_peered":"2024-09-11T05:22:52.726654+0000","last_clean":"2024-09-11T05:22:52.726654+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:52.726654+0000","last_undegraded":"2024-09-11T05:22:52.726654+0000","last_fullsized":"2024-09-11T05:22:52.726654+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268032,"num_objects":1042,"num_object_clones":0,"num_object_copies":2084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18918,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19148","reported_seq":19288,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:53.132936+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:53.132936+0000","last_peered":"2024-09-11T05:22:53.132936+0000","last_clean":"2024-09-11T05:22:53.132936+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:53.132936+0000","last_undegraded":"2024-09-11T05:22:53.132936+0000","last_fullsized":"2024-09-11T05:22:53.132936+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182016,"num_objects":1021,"num_object_clones":0,"num_object_copies":2042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15982,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18945","reported_seq":26301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:51.438775+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:51.438775+0000","last_peered":"2024-09-11T05:22:51.438775+0000","last_clean":"2024-09-11T05:22:51.438775+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:51.438775+0000","last_undegraded":"2024-09-11T05:22:51.438775+0000","last_fullsized":"2024-09-11T05:22:51.438775+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464640,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15489,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19143","reported_seq":20482,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.191214+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:53.191214+0000","last_peered":"2024-09-11T05:22:53.191214+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:53.191214+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4337664,"num_objects":1059,"num_object_clones":0,"num_object_copies":2118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16308,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18892","reported_seq":19027,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:53.179817+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:53.179817+0000","last_peered":"2024-09-11T05:22:53.179817+0000","last_clean":"2024-09-11T05:22:53.179817+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:53.179817+0000","last_undegraded":"2024-09-11T05:22:53.179817+0000","last_fullsized":"2024-09-11T05:22:53.179817+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":992,"num_object_clones":0,"num_object_copies":1984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16195,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19005","reported_seq":19147,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:52.769713+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:52.769713+0000","last_peered":"2024-09-11T05:22:52.769713+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:52.769713+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3956736,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17073,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18672","reported_seq":18846,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.129108+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:53.129108+0000","last_peered":"2024-09-11T05:22:53.129108+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:53.129108+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096000,"num_objects":1000,"num_object_clones":0,"num_object_copies":2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19004","reported_seq":19132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:52.701529+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:52.701529+0000","last_peered":"2024-09-11T05:22:52.701529+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:52.701529+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4214784,"num_objects":1029,"num_object_clones":0,"num_object_copies":2058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17033,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18659","reported_seq":25755,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.677727+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:52.677727+0000","last_peered":"2024-09-11T05:22:52.677727+0000","last_clean":"2024-09-11T05:22:52.677727+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:52.677727+0000","last_undegraded":"2024-09-11T05:22:52.677727+0000","last_fullsized":"2024-09-11T05:22:52.677727+0000","mapping_epoch":44,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4296704,"num_objects":1049,"num_object_clones":0,"num_object_copies":2098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19159","reported_seq":19301,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.196776+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:53.196776+0000","last_peered":"2024-09-11T05:22:53.196776+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:53.196776+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16192,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18655","reported_seq":18797,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.152593+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:53.152593+0000","last_peered":"2024-09-11T05:22:53.152593+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:53.152593+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4091904,"num_objects":999,"num_object_clones":0,"num_object_copies":1998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18783","reported_seq":18929,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.161580+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:53.161580+0000","last_peered":"2024-09-11T05:22:53.161580+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:53.161580+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4165632,"num_objects":1017,"num_object_clones":0,"num_object_copies":2034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15541,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2527}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18634","reported_seq":18788,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.064827+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:53.064827+0000","last_peered":"2024-09-11T05:22:53.064827+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:53.064827+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916393,"num_objects":957,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2558}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":66429569,"num_objects":16220,"num_object_clones":0,"num_object_copies":32440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265475,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11177,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":308424704,"data_stored":308353938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476136,"kb_used_data":147352,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93895704,"statfs":{"total":96636764160,"available":96149200896,"internally_reserved":0,"allocated":150888448,"data_stored":150608582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502456,"kb_used_data":51496,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93869384,"statfs":{"total":96636764160,"available":96122249216,"internally_reserved":0,"allocated":52731904,"data_stored":52395776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3,3,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215228,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534096,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93837744,"statfs":{"total":96636764160,"available":96089849856,"internally_reserved":0,"allocated":107929600,"data_stored":107649729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106758144,"data_stored":106754665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51978240,"data_stored":51918504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149688320,"data_stored":149680769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:56.584 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:57.343 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:57.344 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:57.393 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-11T05:22:55.632667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":66429588,"num_objects":16222,"num_object_clones":0,"num_object_copies":32444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265477,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11178,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512688,"kb_used_data":304248,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281602832,"statfs":{"total":289910292480,"available":288361299968,"internally_reserved":0,"allocated":311549952,"data_stored":310654087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1237371905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,8,5,10,7,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18505728,"num_objects":-4518,"num_object_clones":0,"num_object_copies":-9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4518,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002642"},"pg_stats":[{"pgid":"2.f","version":"49'19172","reported_seq":19261,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.198543+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:53.198543+0000","last_peered":"2024-09-11T05:22:53.198543+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:53.198543+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4169728,"num_objects":1018,"num_object_clones":0,"num_object_copies":2036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19172,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2657}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18952","reported_seq":19064,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.775534+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:52.775534+0000","last_peered":"2024-09-11T05:22:52.775534+0000","last_clean":"2024-09-11T05:22:52.775534+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:52.775534+0000","last_undegraded":"2024-09-11T05:22:52.775534+0000","last_fullsized":"2024-09-11T05:22:52.775534+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17350,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18670","reported_seq":25807,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:51.390427+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:51.390427+0000","last_peered":"2024-09-11T05:22:51.390427+0000","last_clean":"2024-09-11T05:22:51.390427+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:51.390427+0000","last_undegraded":"2024-09-11T05:22:51.390427+0000","last_fullsized":"2024-09-11T05:22:51.390427+0000","mapping_epoch":48,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4288536,"num_objects":1048,"num_object_clones":0,"num_object_copies":2096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17073,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18918","reported_seq":19000,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.726654+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:52.726654+0000","last_peered":"2024-09-11T05:22:52.726654+0000","last_clean":"2024-09-11T05:22:52.726654+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:52.726654+0000","last_undegraded":"2024-09-11T05:22:52.726654+0000","last_fullsized":"2024-09-11T05:22:52.726654+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268032,"num_objects":1042,"num_object_clones":0,"num_object_copies":2084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18918,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19148","reported_seq":19288,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:53.132936+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:53.132936+0000","last_peered":"2024-09-11T05:22:53.132936+0000","last_clean":"2024-09-11T05:22:53.132936+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:53.132936+0000","last_undegraded":"2024-09-11T05:22:53.132936+0000","last_fullsized":"2024-09-11T05:22:53.132936+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182016,"num_objects":1021,"num_object_clones":0,"num_object_copies":2042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15982,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18945","reported_seq":26301,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:51.438775+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:51.438775+0000","last_peered":"2024-09-11T05:22:51.438775+0000","last_clean":"2024-09-11T05:22:51.438775+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:51.438775+0000","last_undegraded":"2024-09-11T05:22:51.438775+0000","last_fullsized":"2024-09-11T05:22:51.438775+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464640,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15489,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19143","reported_seq":20482,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.191214+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:53.191214+0000","last_peered":"2024-09-11T05:22:53.191214+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:53.191214+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4337664,"num_objects":1059,"num_object_clones":0,"num_object_copies":2118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16308,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18892","reported_seq":19027,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:53.179817+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:53.179817+0000","last_peered":"2024-09-11T05:22:53.179817+0000","last_clean":"2024-09-11T05:22:53.179817+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:53.179817+0000","last_undegraded":"2024-09-11T05:22:53.179817+0000","last_fullsized":"2024-09-11T05:22:53.179817+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":992,"num_object_clones":0,"num_object_copies":1984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16195,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19005","reported_seq":19147,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:52.769713+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:52.769713+0000","last_peered":"2024-09-11T05:22:52.769713+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:52.769713+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3956736,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17073,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18672","reported_seq":18846,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.129108+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:53.129108+0000","last_peered":"2024-09-11T05:22:53.129108+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:53.129108+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096000,"num_objects":1000,"num_object_clones":0,"num_object_copies":2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19004","reported_seq":19132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:52.701529+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:52.701529+0000","last_peered":"2024-09-11T05:22:52.701529+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:52.701529+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4214784,"num_objects":1029,"num_object_clones":0,"num_object_copies":2058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17033,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18659","reported_seq":25755,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.677727+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:52.677727+0000","last_peered":"2024-09-11T05:22:52.677727+0000","last_clean":"2024-09-11T05:22:52.677727+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:52.677727+0000","last_undegraded":"2024-09-11T05:22:52.677727+0000","last_fullsized":"2024-09-11T05:22:52.677727+0000","mapping_epoch":44,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4296704,"num_objects":1049,"num_object_clones":0,"num_object_copies":2098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19159","reported_seq":19301,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.196776+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:53.196776+0000","last_peered":"2024-09-11T05:22:53.196776+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:53.196776+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16192,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18655","reported_seq":18797,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.152593+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:53.152593+0000","last_peered":"2024-09-11T05:22:53.152593+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:53.152593+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4091904,"num_objects":999,"num_object_clones":0,"num_object_copies":1998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18783","reported_seq":18929,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.161580+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:53.161580+0000","last_peered":"2024-09-11T05:22:53.161580+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:53.161580+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4165632,"num_objects":1017,"num_object_clones":0,"num_object_copies":2034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15541,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2527}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750539+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750539+0000","last_peered":"2024-09-11T05:22:17.750539+0000","last_clean":"2024-09-11T05:22:17.750539+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750539+0000","last_undegraded":"2024-09-11T05:22:17.750539+0000","last_fullsized":"2024-09-11T05:22:17.750539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18634","reported_seq":18788,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.064827+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:53.064827+0000","last_peered":"2024-09-11T05:22:53.064827+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:53.064827+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916393,"num_objects":957,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2558}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":66429569,"num_objects":16220,"num_object_clones":0,"num_object_copies":32440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265475,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11177,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":308424704,"data_stored":308353938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476136,"kb_used_data":147352,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93895704,"statfs":{"total":96636764160,"available":96149200896,"internally_reserved":0,"allocated":150888448,"data_stored":150608582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502456,"kb_used_data":51496,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93869384,"statfs":{"total":96636764160,"available":96122249216,"internally_reserved":0,"allocated":52731904,"data_stored":52395776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3,3,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215228,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534096,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93837744,"statfs":{"total":96636764160,"available":96089849856,"internally_reserved":0,"allocated":107929600,"data_stored":107649729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106758144,"data_stored":106754665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51978240,"data_stored":51918504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149688320,"data_stored":149680769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:22:57.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:22:57.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19172", 'reported_seq': 19261, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:53.198543+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:22:53.198543+0000', 'last_peered': '2024-09-11T05:22:53.198543+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:22:53.198543+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4169728, 'num_objects': 1018, 'num_object_clones': 0, 'num_object_copies': 2036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19172, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2657}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:57.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:22:57.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19143", 'reported_seq': 20482, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:53.191214+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:22:53.191214+0000', 'last_peered': '2024-09-11T05:22:53.191214+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:22:53.191214+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4337664, 'num_objects': 1059, 'num_object_clones': 0, 'num_object_copies': 2118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2979, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16308, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 599, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1920}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19005", 'reported_seq': 19147, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:52.769713+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:22:52.769713+0000', 'last_peered': '2024-09-11T05:22:52.769713+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:22:52.769713+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3956736, 'num_objects': 966, 'num_object_clones': 0, 'num_object_copies': 1932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17073, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2559}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18672", 'reported_seq': 18846, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:53.129108+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:22:53.129108+0000', 'last_peered': '2024-09-11T05:22:53.129108+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:22:53.129108+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4096000, 'num_objects': 1000, 'num_object_clones': 0, 'num_object_copies': 2000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1000, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19004", 'reported_seq': 19132, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:52.701529+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:22:52.701529+0000', 'last_peered': '2024-09-11T05:22:52.701529+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:22:52.701529+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4214784, 'num_objects': 1029, 'num_object_clones': 0, 'num_object_copies': 2058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17033, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2493}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19159", 'reported_seq': 19301, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:53.196776+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:22:53.196776+0000', 'last_peered': '2024-09-11T05:22:53.196776+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:22:53.196776+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4034560, 'num_objects': 985, 'num_object_clones': 0, 'num_object_copies': 1970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16192, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:22:57.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18655", 'reported_seq': 18797, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:53.152593+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:22:53.152593+0000', 'last_peered': '2024-09-11T05:22:53.152593+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:22:53.152593+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4091904, 'num_objects': 999, 'num_object_clones': 0, 'num_object_copies': 1998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15521, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2496}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:57.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:22:57.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18783", 'reported_seq': 18929, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:53.161580+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:22:53.161580+0000', 'last_peered': '2024-09-11T05:22:53.161580+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:22:53.161580+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4165632, 'num_objects': 1017, 'num_object_clones': 0, 'num_object_copies': 2034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15541, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2527}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:57.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:22:57.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18634", 'reported_seq': 18788, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:22:53.064827+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:22:53.064827+0000', 'last_peered': '2024-09-11T05:22:53.064827+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:22:53.064827+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3916393, 'num_objects': 957, 'num_object_clones': 0, 'num_object_copies': 1914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 957, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15641, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2558}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:22:57.396 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:22:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:58.438 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31335/31898 objects degraded (98.235%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":656,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":15949,"data_bytes":65311380,"bytes_used":1550438400,"bytes_avail":288359854080,"bytes_total":289910292480,"degraded_objects":31335,"degraded_total":31898,"degraded_ratio":0.98234999059502159,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":399},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:22:58.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:22:58.501 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:22:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:22:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:22:59.269 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-11T05:22:57.633114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":65311380,"num_objects":15949,"num_object_clones":0,"num_object_copies":31898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265750,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11178,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514100,"kb_used_data":297336,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281601420,"statfs":{"total":289910292480,"available":288359854080,"internally_reserved":0,"allocated":304472064,"data_stored":303576199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1245891585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,7,5,11,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19623936,"num_objects":-4791,"num_object_clones":0,"num_object_copies":-9582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4791,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002616"},"pg_stats":[{"pgid":"2.f","version":"49'19172","reported_seq":19261,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.198543+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:53.198543+0000","last_peered":"2024-09-11T05:22:53.198543+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:53.198543+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4169728,"num_objects":1018,"num_object_clones":0,"num_object_copies":2036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19172,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2657}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18952","reported_seq":19064,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.775534+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:52.775534+0000","last_peered":"2024-09-11T05:22:52.775534+0000","last_clean":"2024-09-11T05:22:52.775534+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:52.775534+0000","last_undegraded":"2024-09-11T05:22:52.775534+0000","last_fullsized":"2024-09-11T05:22:52.775534+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17350,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18803","reported_seq":25940,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:56.367707+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:56.367707+0000","last_peered":"2024-09-11T05:22:56.367707+0000","last_clean":"2024-09-11T05:22:56.367707+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:56.367707+0000","last_undegraded":"2024-09-11T05:22:56.367707+0000","last_fullsized":"2024-09-11T05:22:56.367707+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3743768,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17206,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'18918","reported_seq":19000,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.726654+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:52.726654+0000","last_peered":"2024-09-11T05:22:52.726654+0000","last_clean":"2024-09-11T05:22:52.726654+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:52.726654+0000","last_undegraded":"2024-09-11T05:22:52.726654+0000","last_fullsized":"2024-09-11T05:22:52.726654+0000","mapping_epoch":35,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4268032,"num_objects":1042,"num_object_clones":0,"num_object_copies":2084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18918,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19148","reported_seq":19288,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:53.132936+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:53.132936+0000","last_peered":"2024-09-11T05:22:53.132936+0000","last_clean":"2024-09-11T05:22:53.132936+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:53.132936+0000","last_undegraded":"2024-09-11T05:22:53.132936+0000","last_fullsized":"2024-09-11T05:22:53.132936+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182016,"num_objects":1021,"num_object_clones":0,"num_object_copies":2042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15982,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19085","reported_seq":26441,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:56.416148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:56.416148+0000","last_peered":"2024-09-11T05:22:56.416148+0000","last_clean":"2024-09-11T05:22:56.416148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:56.416148+0000","last_undegraded":"2024-09-11T05:22:56.416148+0000","last_fullsized":"2024-09-11T05:22:56.416148+0000","mapping_epoch":46,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3891200,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15629,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19143","reported_seq":20482,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.191214+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:53.191214+0000","last_peered":"2024-09-11T05:22:53.191214+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:53.191214+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4337664,"num_objects":1059,"num_object_clones":0,"num_object_copies":2118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16308,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18892","reported_seq":19027,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:53.179817+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:53.179817+0000","last_peered":"2024-09-11T05:22:53.179817+0000","last_clean":"2024-09-11T05:22:53.179817+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:53.179817+0000","last_undegraded":"2024-09-11T05:22:53.179817+0000","last_fullsized":"2024-09-11T05:22:53.179817+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":992,"num_object_clones":0,"num_object_copies":1984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16195,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19005","reported_seq":19147,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:52.769713+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:52.769713+0000","last_peered":"2024-09-11T05:22:52.769713+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:52.769713+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3956736,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17073,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18672","reported_seq":18846,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.129108+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:53.129108+0000","last_peered":"2024-09-11T05:22:53.129108+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:53.129108+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096000,"num_objects":1000,"num_object_clones":0,"num_object_copies":2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19004","reported_seq":19132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:52.701529+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:52.701529+0000","last_peered":"2024-09-11T05:22:52.701529+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:52.701529+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4214784,"num_objects":1029,"num_object_clones":0,"num_object_copies":2058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17033,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18659","reported_seq":25755,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:52.677727+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:52.677727+0000","last_peered":"2024-09-11T05:22:52.677727+0000","last_clean":"2024-09-11T05:22:52.677727+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:52.677727+0000","last_undegraded":"2024-09-11T05:22:52.677727+0000","last_fullsized":"2024-09-11T05:22:52.677727+0000","mapping_epoch":44,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4296704,"num_objects":1049,"num_object_clones":0,"num_object_copies":2098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19159","reported_seq":19301,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.196776+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:53.196776+0000","last_peered":"2024-09-11T05:22:53.196776+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:53.196776+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16192,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18655","reported_seq":18797,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.152593+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:53.152593+0000","last_peered":"2024-09-11T05:22:53.152593+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:53.152593+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4091904,"num_objects":999,"num_object_clones":0,"num_object_copies":1998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18783","reported_seq":18929,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.161580+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:53.161580+0000","last_peered":"2024-09-11T05:22:53.161580+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:53.161580+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4165632,"num_objects":1017,"num_object_clones":0,"num_object_copies":2034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15541,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2527}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18634","reported_seq":18788,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:53.064827+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:53.064827+0000","last_peered":"2024-09-11T05:22:53.064827+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:53.064827+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3916393,"num_objects":957,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2558}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":65311361,"num_objects":15947,"num_object_clones":0,"num_object_copies":31894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265748,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11177,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":301367296,"data_stored":301296530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476136,"kb_used_data":147352,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93895704,"statfs":{"total":96636764160,"available":96149200896,"internally_reserved":0,"allocated":150888448,"data_stored":150608582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503868,"kb_used_data":44584,"kb_used_omap":4,"kb_used_meta":459259,"kb_avail":93867972,"statfs":{"total":96636764160,"available":96120803328,"internally_reserved":0,"allocated":45654016,"data_stored":45317888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470281899},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,3,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215228,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534096,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93837744,"statfs":{"total":96636764160,"available":96089849856,"internally_reserved":0,"allocated":107929600,"data_stored":107649729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106758144,"data_stored":106754665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44920832,"data_stored":44861096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149688320,"data_stored":149680769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:02.271 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:03.231 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:03.234 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:03.261 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-11T05:23:01.634058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":57021076,"num_objects":13925,"num_object_clones":0,"num_object_copies":27850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267774,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11257,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501872,"kb_used_data":285112,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281613648,"statfs":{"total":289910292480,"available":288372375552,"internally_reserved":0,"allocated":291954688,"data_stored":291058823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1245891585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,1,6,7,8,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18649088,"num_objects":-4553,"num_object_clones":0,"num_object_copies":-9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4553,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002675"},"pg_stats":[{"pgid":"2.f","version":"49'19315","reported_seq":19402,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:58.238526+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:22:58.238526+0000","last_peered":"2024-09-11T05:22:58.238526+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:22:58.238526+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3584000,"num_objects":875,"num_object_clones":0,"num_object_copies":1750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19315,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2800}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19093","reported_seq":19204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:57.807307+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:22:57.807307+0000","last_peered":"2024-09-11T05:22:57.807307+0000","last_clean":"2024-09-11T05:22:57.807307+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:22:57.807307+0000","last_undegraded":"2024-09-11T05:22:57.807307+0000","last_fullsized":"2024-09-11T05:22:57.807307+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3305472,"num_objects":807,"num_object_clones":0,"num_object_copies":1614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17491,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18803","reported_seq":25940,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:56.367707+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:22:56.367707+0000","last_peered":"2024-09-11T05:22:56.367707+0000","last_clean":"2024-09-11T05:22:56.367707+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:22:56.367707+0000","last_undegraded":"2024-09-11T05:22:56.367707+0000","last_fullsized":"2024-09-11T05:22:56.367707+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3743768,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17206,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19056","reported_seq":19138,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:57.809104+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:22:57.809104+0000","last_peered":"2024-09-11T05:22:57.809104+0000","last_clean":"2024-09-11T05:22:57.809104+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:22:57.809104+0000","last_undegraded":"2024-09-11T05:22:57.809104+0000","last_fullsized":"2024-09-11T05:22:57.809104+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3702784,"num_objects":904,"num_object_clones":0,"num_object_copies":1808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19056,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19313","reported_seq":19453,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:58.170573+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:22:58.170573+0000","last_peered":"2024-09-11T05:22:58.170573+0000","last_clean":"2024-09-11T05:22:58.170573+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:22:58.170573+0000","last_undegraded":"2024-09-11T05:22:58.170573+0000","last_fullsized":"2024-09-11T05:22:58.170573+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3506176,"num_objects":856,"num_object_clones":0,"num_object_copies":1712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19085","reported_seq":26441,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:56.416148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:22:56.416148+0000","last_peered":"2024-09-11T05:22:56.416148+0000","last_clean":"2024-09-11T05:22:56.416148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:22:56.416148+0000","last_undegraded":"2024-09-11T05:22:56.416148+0000","last_fullsized":"2024-09-11T05:22:56.416148+0000","mapping_epoch":46,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3891200,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15629,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19281","reported_seq":20781,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:58.208955+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:22:58.208955+0000","last_peered":"2024-09-11T05:22:58.208955+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:22:58.208955+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3772416,"num_objects":921,"num_object_clones":0,"num_object_copies":1842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16446,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19031","reported_seq":19166,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:58.173929+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:22:58.173929+0000","last_peered":"2024-09-11T05:22:58.173929+0000","last_clean":"2024-09-11T05:22:58.173929+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:22:58.173929+0000","last_undegraded":"2024-09-11T05:22:58.173929+0000","last_fullsized":"2024-09-11T05:22:58.173929+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3493888,"num_objects":853,"num_object_clones":0,"num_object_copies":1706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19151","reported_seq":19293,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:57.782340+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:22:57.782340+0000","last_peered":"2024-09-11T05:22:57.782340+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:22:57.782340+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3358720,"num_objects":820,"num_object_clones":0,"num_object_copies":1640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17219,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18811","reported_seq":18987,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:58.222098+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:22:58.222098+0000","last_peered":"2024-09-11T05:22:58.222098+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:22:58.222098+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3526656,"num_objects":861,"num_object_clones":0,"num_object_copies":1722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15564,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2581}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19150","reported_seq":19278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:57.724206+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:22:57.724206+0000","last_peered":"2024-09-11T05:22:57.724206+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:22:57.724206+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3616768,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17179,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2639}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18813","reported_seq":25908,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:57.783847+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:22:57.783847+0000","last_peered":"2024-09-11T05:22:57.783847+0000","last_clean":"2024-09-11T05:22:57.783847+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:22:57.783847+0000","last_undegraded":"2024-09-11T05:22:57.783847+0000","last_fullsized":"2024-09-11T05:22:57.783847+0000","mapping_epoch":44,"log_start":"45'8800","ondisk_log_start":"45'8800","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3665920,"num_objects":895,"num_object_clones":0,"num_object_copies":1790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16716,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19301","reported_seq":19441,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:58.231411+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:22:58.231411+0000","last_peered":"2024-09-11T05:22:58.231411+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:22:58.231411+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3452928,"num_objects":843,"num_object_clones":0,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18798","reported_seq":18942,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:58.224189+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:22:58.224189+0000","last_peered":"2024-09-11T05:22:58.224189+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:22:58.224189+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3506176,"num_objects":856,"num_object_clones":0,"num_object_copies":1712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15664,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2639}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18940","reported_seq":19085,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:58.251591+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:22:58.251591+0000","last_peered":"2024-09-11T05:22:58.251591+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:22:58.251591+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3522560,"num_objects":860,"num_object_clones":0,"num_object_copies":1720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15698,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2684}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18767","reported_seq":18921,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:22:58.102863+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:22:58.102863+0000","last_peered":"2024-09-11T05:22:58.102863+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:22:58.102863+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371625,"num_objects":824,"num_object_clones":0,"num_object_copies":1648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15774,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":57021057,"num_objects":13923,"num_object_clones":0,"num_object_copies":27846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267772,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11256,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":289632256,"data_stored":289561490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462492,"kb_used_data":142028,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93909348,"statfs":{"total":96636764160,"available":96163172352,"internally_reserved":0,"allocated":145436672,"data_stored":145156806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503868,"kb_used_data":44584,"kb_used_omap":4,"kb_used_meta":459259,"kb_avail":93867972,"statfs":{"total":96636764160,"available":96120803328,"internally_reserved":0,"allocated":45654016,"data_stored":45317888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470281899},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,3,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215229,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535512,"kb_used_data":98500,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93836328,"statfs":{"total":96636764160,"available":96088399872,"internally_reserved":0,"allocated":100864000,"data_stored":100584129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,3,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99778560,"data_stored":99775081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44920832,"data_stored":44861096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":144932864,"data_stored":144925313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:03.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:04.103 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:04.229 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-11T05:23:03.636218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":47538836,"num_objects":11610,"num_object_clones":0,"num_object_copies":23220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270089,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11326,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160726,"ondisk_log_size":160726,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480892,"kb_used_data":266892,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281634628,"statfs":{"total":289910292480,"available":288393859072,"internally_reserved":0,"allocated":273297408,"data_stored":272401543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1243073537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,7,2,7,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26980352,"num_objects":-6587,"num_object_clones":0,"num_object_copies":-13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6587,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004470"},"pg_stats":[{"pgid":"2.f","version":"49'19445","reported_seq":19534,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.196830+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:03.196830+0000","last_peered":"2024-09-11T05:23:03.196830+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:03.196830+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3051520,"num_objects":745,"num_object_clones":0,"num_object_copies":1490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19445,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2930}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19227","reported_seq":19340,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:02.810776+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:02.810776+0000","last_peered":"2024-09-11T05:23:02.810776+0000","last_clean":"2024-09-11T05:23:02.810776+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:02.810776+0000","last_undegraded":"2024-09-11T05:23:02.810776+0000","last_fullsized":"2024-09-11T05:23:02.810776+0000","mapping_epoch":35,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2756608,"num_objects":673,"num_object_clones":0,"num_object_copies":1346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17625,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18949","reported_seq":26084,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:01.631529+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:01.631529+0000","last_peered":"2024-09-11T05:23:01.631529+0000","last_clean":"2024-09-11T05:23:01.631529+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:01.631529+0000","last_undegraded":"2024-09-11T05:23:01.631529+0000","last_fullsized":"2024-09-11T05:23:01.631529+0000","mapping_epoch":48,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145752,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17352,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19194","reported_seq":19276,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:02.818489+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:02.818489+0000","last_peered":"2024-09-11T05:23:02.818489+0000","last_clean":"2024-09-11T05:23:02.818489+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:02.818489+0000","last_undegraded":"2024-09-11T05:23:02.818489+0000","last_fullsized":"2024-09-11T05:23:02.818489+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19194,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19466","reported_seq":19606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:03.239374+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:03.239374+0000","last_peered":"2024-09-11T05:23:03.239374+0000","last_clean":"2024-09-11T05:23:03.239374+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:03.239374+0000","last_undegraded":"2024-09-11T05:23:03.239374+0000","last_fullsized":"2024-09-11T05:23:03.239374+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2879488,"num_objects":703,"num_object_clones":0,"num_object_copies":1406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16300,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19222","reported_seq":26578,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:01.546353+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:01.546353+0000","last_peered":"2024-09-11T05:23:01.546353+0000","last_clean":"2024-09-11T05:23:01.546353+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:01.546353+0000","last_undegraded":"2024-09-11T05:23:01.546353+0000","last_fullsized":"2024-09-11T05:23:01.546353+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3330048,"num_objects":813,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15766,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19431","reported_seq":21069,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.227710+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:03.227710+0000","last_peered":"2024-09-11T05:23:03.227710+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:03.227710+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3158016,"num_objects":771,"num_object_clones":0,"num_object_copies":1542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2060}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19177","reported_seq":19312,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:03.201428+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:03.201428+0000","last_peered":"2024-09-11T05:23:03.201428+0000","last_clean":"2024-09-11T05:23:03.201428+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:03.201428+0000","last_undegraded":"2024-09-11T05:23:03.201428+0000","last_fullsized":"2024-09-11T05:23:03.201428+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2895872,"num_objects":707,"num_object_clones":0,"num_object_copies":1414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16480,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19314","reported_seq":19456,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:02.716902+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:02.716902+0000","last_peered":"2024-09-11T05:23:02.716902+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:02.716902+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2691072,"num_objects":657,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17382,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18953","reported_seq":19127,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.255310+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:03.255310+0000","last_peered":"2024-09-11T05:23:03.255310+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:03.255310+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2945024,"num_objects":719,"num_object_clones":0,"num_object_copies":1438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15706,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2723}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19309","reported_seq":19437,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:02.763559+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:02.763559+0000","last_peered":"2024-09-11T05:23:02.763559+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:02.763559+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2965504,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17338,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2798}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18960","reported_seq":26056,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:02.733781+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:02.733781+0000","last_peered":"2024-09-11T05:23:02.733781+0000","last_clean":"2024-09-11T05:23:02.733781+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:02.733781+0000","last_undegraded":"2024-09-11T05:23:02.733781+0000","last_fullsized":"2024-09-11T05:23:02.733781+0000","mapping_epoch":44,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3063808,"num_objects":748,"num_object_clones":0,"num_object_copies":1496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16863,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19444","reported_seq":19586,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.202851+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:03.202851+0000","last_peered":"2024-09-11T05:23:03.202851+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:03.202851+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2867200,"num_objects":700,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18927","reported_seq":19071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.145364+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:03.145364+0000","last_peered":"2024-09-11T05:23:03.145364+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:03.145364+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977792,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19079","reported_seq":19225,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.168256+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:03.168256+0000","last_peered":"2024-09-11T05:23:03.168256+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:03.168256+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2953216,"num_objects":721,"num_object_clones":0,"num_object_copies":1442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15837,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18926","reported_seq":19078,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.225062+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:03.225062+0000","last_peered":"2024-09-11T05:23:03.225062+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:03.225062+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2720361,"num_objects":665,"num_object_clones":0,"num_object_copies":1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15933,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":47538817,"num_objects":11608,"num_object_clones":0,"num_object_copies":23216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270087,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11325,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270680064,"data_stored":270609298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461912,"kb_used_data":137288,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93909928,"statfs":{"total":96636764160,"available":96163766272,"internally_reserved":0,"allocated":140582912,"data_stored":140303046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484544,"kb_used_data":37744,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93887296,"statfs":{"total":96636764160,"available":96140591104,"internally_reserved":0,"allocated":38649856,"data_stored":38313728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215230,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534436,"kb_used_data":91860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93837404,"statfs":{"total":96636764160,"available":96089501696,"internally_reserved":0,"allocated":94064640,"data_stored":93784769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453177003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":92848128,"data_stored":92844649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37728256,"data_stored":37668520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140103680,"data_stored":140096129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19445", 'reported_seq': 19534, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:03.196830+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:03.196830+0000', 'last_peered': '2024-09-11T05:23:03.196830+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:03.196830+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3051520, 'num_objects': 745, 'num_object_clones': 0, 'num_object_copies': 1490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19445, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2930}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19431", 'reported_seq': 21069, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:03.227710+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:03.227710+0000', 'last_peered': '2024-09-11T05:23:03.227710+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:03.227710+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3158016, 'num_objects': 771, 'num_object_clones': 0, 'num_object_copies': 1542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16596, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 747, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2060}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19314", 'reported_seq': 19456, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:02.716902+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:02.716902+0000', 'last_peered': '2024-09-11T05:23:02.716902+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:02.716902+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2691072, 'num_objects': 657, 'num_object_clones': 0, 'num_object_copies': 1314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17382, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:04.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18953", 'reported_seq': 19127, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:03.255310+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:03.255310+0000', 'last_peered': '2024-09-11T05:23:03.255310+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:03.255310+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2945024, 'num_objects': 719, 'num_object_clones': 0, 'num_object_copies': 1438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 719, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15706, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2723}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19309", 'reported_seq': 19437, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:02.763559+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:02.763559+0000', 'last_peered': '2024-09-11T05:23:02.763559+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:02.763559+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2965504, 'num_objects': 724, 'num_object_clones': 0, 'num_object_copies': 1448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 724, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17338, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2798}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19444", 'reported_seq': 19586, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:03.202851+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:03.202851+0000', 'last_peered': '2024-09-11T05:23:03.202851+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:03.202851+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2867200, 'num_objects': 700, 'num_object_clones': 0, 'num_object_copies': 1400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16477, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2923}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'18927", 'reported_seq': 19071, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:03.145364+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:03.145364+0000', 'last_peered': '2024-09-11T05:23:03.145364+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:03.145364+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2977792, 'num_objects': 727, 'num_object_clones': 0, 'num_object_copies': 1454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15793, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19079", 'reported_seq': 19225, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:03.168256+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:03.168256+0000', 'last_peered': '2024-09-11T05:23:03.168256+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:03.168256+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2953216, 'num_objects': 721, 'num_object_clones': 0, 'num_object_copies': 1442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15837, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2823}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:04.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18926", 'reported_seq': 19078, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:03.225062+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:03.225062+0000', 'last_peered': '2024-09-11T05:23:03.225062+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:03.225062+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2720361, 'num_objects': 665, 'num_object_clones': 0, 'num_object_copies': 1330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15933, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2850}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:04.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:05.271 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:05.272 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31187/23220 objects degraded (134.311%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":663,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":11610,"data_bytes":47538836,"bytes_used":1516433408,"bytes_avail":288393859072,"bytes_total":289910292480,"degraded_objects":31187,"degraded_total":23220,"degraded_ratio":1.3431093884582257,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":548},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:05.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:05.310 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:06.218 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:06.219 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:06.304 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-11T05:23:05.638710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":47538836,"num_objects":11610,"num_object_clones":0,"num_object_copies":23220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270089,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11326,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160726,"ondisk_log_size":160726,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480892,"kb_used_data":266892,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281634628,"statfs":{"total":289910292480,"available":288393859072,"internally_reserved":0,"allocated":273297408,"data_stored":272401543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1243073537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,7,2,7,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18890752,"num_objects":-4612,"num_object_clones":0,"num_object_copies":-9224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006507"},"pg_stats":[{"pgid":"2.f","version":"49'19445","reported_seq":19534,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.196830+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:03.196830+0000","last_peered":"2024-09-11T05:23:03.196830+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:03.196830+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3051520,"num_objects":745,"num_object_clones":0,"num_object_copies":1490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19445,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2930}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19227","reported_seq":19340,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:02.810776+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:02.810776+0000","last_peered":"2024-09-11T05:23:02.810776+0000","last_clean":"2024-09-11T05:23:02.810776+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:02.810776+0000","last_undegraded":"2024-09-11T05:23:02.810776+0000","last_fullsized":"2024-09-11T05:23:02.810776+0000","mapping_epoch":35,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2756608,"num_objects":673,"num_object_clones":0,"num_object_copies":1346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17625,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18949","reported_seq":26084,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:01.631529+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:01.631529+0000","last_peered":"2024-09-11T05:23:01.631529+0000","last_clean":"2024-09-11T05:23:01.631529+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:01.631529+0000","last_undegraded":"2024-09-11T05:23:01.631529+0000","last_fullsized":"2024-09-11T05:23:01.631529+0000","mapping_epoch":48,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145752,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17352,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19194","reported_seq":19276,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:02.818489+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:02.818489+0000","last_peered":"2024-09-11T05:23:02.818489+0000","last_clean":"2024-09-11T05:23:02.818489+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:02.818489+0000","last_undegraded":"2024-09-11T05:23:02.818489+0000","last_fullsized":"2024-09-11T05:23:02.818489+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19194,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19466","reported_seq":19606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:03.239374+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:03.239374+0000","last_peered":"2024-09-11T05:23:03.239374+0000","last_clean":"2024-09-11T05:23:03.239374+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:03.239374+0000","last_undegraded":"2024-09-11T05:23:03.239374+0000","last_fullsized":"2024-09-11T05:23:03.239374+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2879488,"num_objects":703,"num_object_clones":0,"num_object_copies":1406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16300,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19222","reported_seq":26578,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:01.546353+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:01.546353+0000","last_peered":"2024-09-11T05:23:01.546353+0000","last_clean":"2024-09-11T05:23:01.546353+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:01.546353+0000","last_undegraded":"2024-09-11T05:23:01.546353+0000","last_fullsized":"2024-09-11T05:23:01.546353+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3330048,"num_objects":813,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15766,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19431","reported_seq":21069,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.227710+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:03.227710+0000","last_peered":"2024-09-11T05:23:03.227710+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:03.227710+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3158016,"num_objects":771,"num_object_clones":0,"num_object_copies":1542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2060}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19177","reported_seq":19312,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:03.201428+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:03.201428+0000","last_peered":"2024-09-11T05:23:03.201428+0000","last_clean":"2024-09-11T05:23:03.201428+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:03.201428+0000","last_undegraded":"2024-09-11T05:23:03.201428+0000","last_fullsized":"2024-09-11T05:23:03.201428+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2895872,"num_objects":707,"num_object_clones":0,"num_object_copies":1414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16480,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19314","reported_seq":19456,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:02.716902+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:02.716902+0000","last_peered":"2024-09-11T05:23:02.716902+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:02.716902+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2691072,"num_objects":657,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17382,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18953","reported_seq":19127,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.255310+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:03.255310+0000","last_peered":"2024-09-11T05:23:03.255310+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:03.255310+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2945024,"num_objects":719,"num_object_clones":0,"num_object_copies":1438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15706,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2723}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19309","reported_seq":19437,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:02.763559+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:02.763559+0000","last_peered":"2024-09-11T05:23:02.763559+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:02.763559+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2965504,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17338,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2798}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18960","reported_seq":26056,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:02.733781+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:02.733781+0000","last_peered":"2024-09-11T05:23:02.733781+0000","last_clean":"2024-09-11T05:23:02.733781+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:02.733781+0000","last_undegraded":"2024-09-11T05:23:02.733781+0000","last_fullsized":"2024-09-11T05:23:02.733781+0000","mapping_epoch":44,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3063808,"num_objects":748,"num_object_clones":0,"num_object_copies":1496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16863,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19444","reported_seq":19586,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.202851+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:03.202851+0000","last_peered":"2024-09-11T05:23:03.202851+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:03.202851+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2867200,"num_objects":700,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18927","reported_seq":19071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.145364+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:03.145364+0000","last_peered":"2024-09-11T05:23:03.145364+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:03.145364+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977792,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19079","reported_seq":19225,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.168256+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:03.168256+0000","last_peered":"2024-09-11T05:23:03.168256+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:03.168256+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2953216,"num_objects":721,"num_object_clones":0,"num_object_copies":1442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15837,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18926","reported_seq":19078,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:03.225062+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:03.225062+0000","last_peered":"2024-09-11T05:23:03.225062+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:03.225062+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2720361,"num_objects":665,"num_object_clones":0,"num_object_copies":1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15933,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":47538817,"num_objects":11608,"num_object_clones":0,"num_object_copies":23216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270087,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11325,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270680064,"data_stored":270609298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461912,"kb_used_data":137288,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93909928,"statfs":{"total":96636764160,"available":96163766272,"internally_reserved":0,"allocated":140582912,"data_stored":140303046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332394155},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484544,"kb_used_data":37744,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93887296,"statfs":{"total":96636764160,"available":96140591104,"internally_reserved":0,"allocated":38649856,"data_stored":38313728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215230,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534436,"kb_used_data":91860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93837404,"statfs":{"total":96636764160,"available":96089501696,"internally_reserved":0,"allocated":94064640,"data_stored":93784769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453177003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":92848128,"data_stored":92844649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37728256,"data_stored":37668520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":140103680,"data_stored":140096129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:09.307 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:11.256 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:11.259 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:11.325 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-11T05:23:09.643006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38408852,"num_objects":9381,"num_object_clones":0,"num_object_copies":18762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272318,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11414,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160955,"ondisk_log_size":160955,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483800,"kb_used_data":248996,"kb_used_omap":12,"kb_used_meta":1234739,"kb_avail":281631720,"statfs":{"total":289910292480,"available":288390881280,"internally_reserved":0,"allocated":254971904,"data_stored":254076039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1264372737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,14,5,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26902528,"num_objects":-6568,"num_object_clones":0,"num_object_copies":-13136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6568,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009892"},"pg_stats":[{"pgid":"2.f","version":"49'19613","reported_seq":19702,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.228487+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:08.228487+0000","last_peered":"2024-09-11T05:23:08.228487+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:08.228487+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2363392,"num_objects":577,"num_object_clones":0,"num_object_copies":1154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19613,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3098}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19342","reported_seq":19455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:07.759137+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:07.759137+0000","last_peered":"2024-09-11T05:23:07.759137+0000","last_clean":"2024-09-11T05:23:07.759137+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:07.759137+0000","last_undegraded":"2024-09-11T05:23:07.759137+0000","last_fullsized":"2024-09-11T05:23:07.759137+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285568,"num_objects":558,"num_object_clones":0,"num_object_copies":1116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17740,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19086","reported_seq":26223,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:06.625423+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:06.625423+0000","last_peered":"2024-09-11T05:23:06.625423+0000","last_clean":"2024-09-11T05:23:06.625423+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:06.625423+0000","last_undegraded":"2024-09-11T05:23:06.625423+0000","last_fullsized":"2024-09-11T05:23:06.625423+0000","mapping_epoch":48,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584600,"num_objects":632,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17489,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19340","reported_seq":19422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:07.742801+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:07.742801+0000","last_peered":"2024-09-11T05:23:07.742801+0000","last_clean":"2024-09-11T05:23:07.742801+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:07.742801+0000","last_undegraded":"2024-09-11T05:23:07.742801+0000","last_fullsized":"2024-09-11T05:23:07.742801+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2539520,"num_objects":620,"num_object_clones":0,"num_object_copies":1240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19340,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19595","reported_seq":19735,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:08.129584+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:08.129584+0000","last_peered":"2024-09-11T05:23:08.129584+0000","last_clean":"2024-09-11T05:23:08.129584+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:08.129584+0000","last_undegraded":"2024-09-11T05:23:08.129584+0000","last_fullsized":"2024-09-11T05:23:08.129584+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2351104,"num_objects":574,"num_object_clones":0,"num_object_copies":1148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16429,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19361","reported_seq":26717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:06.630552+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:06.630552+0000","last_peered":"2024-09-11T05:23:06.630552+0000","last_clean":"2024-09-11T05:23:06.630552+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:06.630552+0000","last_undegraded":"2024-09-11T05:23:06.630552+0000","last_fullsized":"2024-09-11T05:23:06.630552+0000","mapping_epoch":46,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15905,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19572","reported_seq":21386,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.283511+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:08.283511+0000","last_peered":"2024-09-11T05:23:08.283511+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:08.283511+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2580480,"num_objects":630,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":835,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19309","reported_seq":19444,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:08.172101+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:08.172101+0000","last_peered":"2024-09-11T05:23:08.172101+0000","last_clean":"2024-09-11T05:23:08.172101+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:08.172101+0000","last_undegraded":"2024-09-11T05:23:08.172101+0000","last_fullsized":"2024-09-11T05:23:08.172101+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355200,"num_objects":575,"num_object_clones":0,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16612,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19444","reported_seq":19586,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:07.823566+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:07.823566+0000","last_peered":"2024-09-11T05:23:07.823566+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:07.823566+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158592,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17512,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19093","reported_seq":19269,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.153335+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:08.153335+0000","last_peered":"2024-09-11T05:23:08.153335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:08.153335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2371584,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15846,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19449","reported_seq":19577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:07.738391+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:07.738391+0000","last_peered":"2024-09-11T05:23:07.738391+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:07.738391+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2392064,"num_objects":584,"num_object_clones":0,"num_object_copies":1168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17478,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2938}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19095","reported_seq":26191,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:07.771765+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:07.771765+0000","last_peered":"2024-09-11T05:23:07.771765+0000","last_clean":"2024-09-11T05:23:07.771765+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:07.771765+0000","last_undegraded":"2024-09-11T05:23:07.771765+0000","last_fullsized":"2024-09-11T05:23:07.771765+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2510848,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16998,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19593","reported_seq":19735,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.253918+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:08.253918+0000","last_peered":"2024-09-11T05:23:08.253918+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:08.253918+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2256896,"num_objects":551,"num_object_clones":0,"num_object_copies":1102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16626,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19069","reported_seq":19212,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.278714+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:08.278714+0000","last_peered":"2024-09-11T05:23:08.278714+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:08.278714+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2396160,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15935,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2910}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19222","reported_seq":19368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.271472+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:08.271472+0000","last_peered":"2024-09-11T05:23:08.271472+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:08.271472+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367488,"num_objects":578,"num_object_clones":0,"num_object_copies":1156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15980,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19069","reported_seq":19223,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.154306+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:08.154306+0000","last_peered":"2024-09-11T05:23:08.154306+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:08.154306+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2134633,"num_objects":522,"num_object_clones":0,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":38408833,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272316,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11413,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":252198912,"data_stored":252128146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461840,"kb_used_data":133052,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93910000,"statfs":{"total":96636764160,"available":96163840000,"internally_reserved":0,"allocated":136245248,"data_stored":135965382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486064,"kb_used_data":30944,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93885776,"statfs":{"total":96636764160,"available":96139034624,"internally_reserved":0,"allocated":31686656,"data_stored":31350528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215231,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":85000,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":87040000,"data_stored":86760129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":85782528,"data_stored":85779049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30695424,"data_stored":30635688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135720960,"data_stored":135713409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:11.327 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:12.125 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:12.127 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:12.165 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-11T05:23:11.643491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38408852,"num_objects":9381,"num_object_clones":0,"num_object_copies":18762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272318,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11414,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160955,"ondisk_log_size":160955,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483800,"kb_used_data":248996,"kb_used_omap":12,"kb_used_meta":1234739,"kb_avail":281631720,"statfs":{"total":289910292480,"available":288390881280,"internally_reserved":0,"allocated":254971904,"data_stored":254076039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1264372737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,14,5,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18612224,"num_objects":-4544,"num_object_clones":0,"num_object_copies":-9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4544,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009901"},"pg_stats":[{"pgid":"2.f","version":"49'19613","reported_seq":19702,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.228487+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:08.228487+0000","last_peered":"2024-09-11T05:23:08.228487+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:08.228487+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2363392,"num_objects":577,"num_object_clones":0,"num_object_copies":1154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19613,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3098}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19342","reported_seq":19455,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:07.759137+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:07.759137+0000","last_peered":"2024-09-11T05:23:07.759137+0000","last_clean":"2024-09-11T05:23:07.759137+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:07.759137+0000","last_undegraded":"2024-09-11T05:23:07.759137+0000","last_fullsized":"2024-09-11T05:23:07.759137+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2285568,"num_objects":558,"num_object_clones":0,"num_object_copies":1116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17740,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19086","reported_seq":26223,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:06.625423+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:06.625423+0000","last_peered":"2024-09-11T05:23:06.625423+0000","last_clean":"2024-09-11T05:23:06.625423+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:06.625423+0000","last_undegraded":"2024-09-11T05:23:06.625423+0000","last_fullsized":"2024-09-11T05:23:06.625423+0000","mapping_epoch":48,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584600,"num_objects":632,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17489,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19340","reported_seq":19422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:07.742801+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:07.742801+0000","last_peered":"2024-09-11T05:23:07.742801+0000","last_clean":"2024-09-11T05:23:07.742801+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:07.742801+0000","last_undegraded":"2024-09-11T05:23:07.742801+0000","last_fullsized":"2024-09-11T05:23:07.742801+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2539520,"num_objects":620,"num_object_clones":0,"num_object_copies":1240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19340,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19595","reported_seq":19735,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:08.129584+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:08.129584+0000","last_peered":"2024-09-11T05:23:08.129584+0000","last_clean":"2024-09-11T05:23:08.129584+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:08.129584+0000","last_undegraded":"2024-09-11T05:23:08.129584+0000","last_fullsized":"2024-09-11T05:23:08.129584+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2351104,"num_objects":574,"num_object_clones":0,"num_object_copies":1148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16429,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19361","reported_seq":26717,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:06.630552+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:06.630552+0000","last_peered":"2024-09-11T05:23:06.630552+0000","last_clean":"2024-09-11T05:23:06.630552+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:06.630552+0000","last_undegraded":"2024-09-11T05:23:06.630552+0000","last_fullsized":"2024-09-11T05:23:06.630552+0000","mapping_epoch":46,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15905,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19572","reported_seq":21386,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.283511+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:08.283511+0000","last_peered":"2024-09-11T05:23:08.283511+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:08.283511+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2580480,"num_objects":630,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":835,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19309","reported_seq":19444,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:08.172101+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:08.172101+0000","last_peered":"2024-09-11T05:23:08.172101+0000","last_clean":"2024-09-11T05:23:08.172101+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:08.172101+0000","last_undegraded":"2024-09-11T05:23:08.172101+0000","last_fullsized":"2024-09-11T05:23:08.172101+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355200,"num_objects":575,"num_object_clones":0,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16612,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19444","reported_seq":19586,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:07.823566+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:07.823566+0000","last_peered":"2024-09-11T05:23:07.823566+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:07.823566+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158592,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17512,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19093","reported_seq":19269,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.153335+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:08.153335+0000","last_peered":"2024-09-11T05:23:08.153335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:08.153335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2371584,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15846,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19449","reported_seq":19577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:07.738391+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:07.738391+0000","last_peered":"2024-09-11T05:23:07.738391+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:07.738391+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2392064,"num_objects":584,"num_object_clones":0,"num_object_copies":1168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17478,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2938}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19095","reported_seq":26191,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:07.771765+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:07.771765+0000","last_peered":"2024-09-11T05:23:07.771765+0000","last_clean":"2024-09-11T05:23:07.771765+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:07.771765+0000","last_undegraded":"2024-09-11T05:23:07.771765+0000","last_fullsized":"2024-09-11T05:23:07.771765+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2510848,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16998,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19593","reported_seq":19735,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.253918+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:08.253918+0000","last_peered":"2024-09-11T05:23:08.253918+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:08.253918+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2256896,"num_objects":551,"num_object_clones":0,"num_object_copies":1102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16626,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19069","reported_seq":19212,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.278714+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:08.278714+0000","last_peered":"2024-09-11T05:23:08.278714+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:08.278714+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2396160,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15935,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2910}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19222","reported_seq":19368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.271472+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:08.271472+0000","last_peered":"2024-09-11T05:23:08.271472+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:08.271472+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367488,"num_objects":578,"num_object_clones":0,"num_object_copies":1156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15980,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19069","reported_seq":19223,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:08.154306+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:08.154306+0000","last_peered":"2024-09-11T05:23:08.154306+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:08.154306+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2134633,"num_objects":522,"num_object_clones":0,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":38408833,"num_objects":9379,"num_object_clones":0,"num_object_copies":18758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272316,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11413,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":252198912,"data_stored":252128146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461840,"kb_used_data":133052,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93910000,"statfs":{"total":96636764160,"available":96163840000,"internally_reserved":0,"allocated":136245248,"data_stored":135965382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336653995},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486064,"kb_used_data":30944,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93885776,"statfs":{"total":96636764160,"available":96139034624,"internally_reserved":0,"allocated":31686656,"data_stored":31350528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215231,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":85000,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":87040000,"data_stored":86760129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":85782528,"data_stored":85779049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30695424,"data_stored":30635688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135720960,"data_stored":135713409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:12.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:12.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19613", 'reported_seq': 19702, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:08.228487+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:08.228487+0000', 'last_peered': '2024-09-11T05:23:08.228487+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:08.228487+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2363392, 'num_objects': 577, 'num_object_clones': 0, 'num_object_copies': 1154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19613, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3098}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19572", 'reported_seq': 21386, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:08.283511+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:08.283511+0000', 'last_peered': '2024-09-11T05:23:08.283511+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:08.283511+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2580480, 'num_objects': 630, 'num_object_clones': 0, 'num_object_copies': 1260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2743, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16737, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 835, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2113}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19444", 'reported_seq': 19586, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:07.823566+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:07.823566+0000', 'last_peered': '2024-09-11T05:23:07.823566+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:07.823566+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2158592, 'num_objects': 527, 'num_object_clones': 0, 'num_object_copies': 1054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17512, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19093", 'reported_seq': 19269, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:08.153335+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:08.153335+0000', 'last_peered': '2024-09-11T05:23:08.153335+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:08.153335+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2371584, 'num_objects': 579, 'num_object_clones': 0, 'num_object_copies': 1158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 579, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15846, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19449", 'reported_seq': 19577, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:07.738391+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:07.738391+0000', 'last_peered': '2024-09-11T05:23:07.738391+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:07.738391+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2392064, 'num_objects': 584, 'num_object_clones': 0, 'num_object_copies': 1168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17478, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2938}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:12.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:12.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19593", 'reported_seq': 19735, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:08.253918+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:08.253918+0000', 'last_peered': '2024-09-11T05:23:08.253918+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:08.253918+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2256896, 'num_objects': 551, 'num_object_clones': 0, 'num_object_copies': 1102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16626, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3072}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:12.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:12.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19069", 'reported_seq': 19212, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:08.278714+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:08.278714+0000', 'last_peered': '2024-09-11T05:23:08.278714+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:08.278714+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2396160, 'num_objects': 585, 'num_object_clones': 0, 'num_object_copies': 1170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15935, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2910}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:12.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:12.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19222", 'reported_seq': 19368, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:08.271472+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:08.271472+0000', 'last_peered': '2024-09-11T05:23:08.271472+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:08.271472+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2367488, 'num_objects': 578, 'num_object_clones': 0, 'num_object_copies': 1156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15980, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2966}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:12.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:12.168 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19069", 'reported_seq': 19223, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:08.154306+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:08.154306+0000', 'last_peered': '2024-09-11T05:23:08.154306+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:08.154306+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2134633, 'num_objects': 522, 'num_object_clones': 0, 'num_object_copies': 1044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 522, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16076, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2993}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:12.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:13.207 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:13.207 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31099/18762 objects degraded (165.755%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":670,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9381,"data_bytes":38408852,"bytes_used":1519411200,"bytes_avail":288390881280,"bytes_total":289910292480,"degraded_objects":31099,"degraded_total":18762,"degraded_ratio":1.6575524997335038,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:13.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:13.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:14.176 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:14.177 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:14.241 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-11T05:23:13.643902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28959380,"num_objects":7074,"num_object_clones":0,"num_object_copies":14148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274625,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11487,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160562,"ondisk_log_size":160562,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461112,"kb_used_data":230456,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281654408,"statfs":{"total":289910292480,"available":288414113792,"internally_reserved":0,"allocated":235986944,"data_stored":235091079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1260112897},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,14,8,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28061696,"num_objects":-6851,"num_object_clones":0,"num_object_copies":-13702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6851,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009843"},"pg_stats":[{"pgid":"2.f","version":"49'19752","reported_seq":19840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.320949+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:13.320949+0000","last_peered":"2024-09-11T05:23:13.320949+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:13.320949+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1794048,"num_objects":438,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3237}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19482","reported_seq":19595,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.809467+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:12.809467+0000","last_peered":"2024-09-11T05:23:12.809467+0000","last_clean":"2024-09-11T05:23:12.809467+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:12.809467+0000","last_undegraded":"2024-09-11T05:23:12.809467+0000","last_fullsized":"2024-09-11T05:23:12.809467+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712128,"num_objects":418,"num_object_clones":0,"num_object_copies":836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17880,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19222","reported_seq":26359,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:11.710365+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:11.710365+0000","last_peered":"2024-09-11T05:23:11.710365+0000","last_clean":"2024-09-11T05:23:11.710365+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:11.710365+0000","last_undegraded":"2024-09-11T05:23:11.710365+0000","last_fullsized":"2024-09-11T05:23:11.710365+0000","mapping_epoch":48,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2027544,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17625,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19505","reported_seq":19587,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.822494+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:12.822494+0000","last_peered":"2024-09-11T05:23:12.822494+0000","last_clean":"2024-09-11T05:23:12.822494+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:12.822494+0000","last_undegraded":"2024-09-11T05:23:12.822494+0000","last_fullsized":"2024-09-11T05:23:12.822494+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1863680,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19505,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19729","reported_seq":19869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:13.286428+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:13.286428+0000","last_peered":"2024-09-11T05:23:13.286428+0000","last_clean":"2024-09-11T05:23:13.286428+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:13.286428+0000","last_undegraded":"2024-09-11T05:23:13.286428+0000","last_fullsized":"2024-09-11T05:23:13.286428+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1802240,"num_objects":440,"num_object_clones":0,"num_object_copies":880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16563,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19537","reported_seq":26893,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:11.678853+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:11.678853+0000","last_peered":"2024-09-11T05:23:11.678853+0000","last_clean":"2024-09-11T05:23:11.678853+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:11.678853+0000","last_undegraded":"2024-09-11T05:23:11.678853+0000","last_fullsized":"2024-09-11T05:23:11.678853+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2039808,"num_objects":498,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16081,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19726","reported_seq":21686,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.315337+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:13.315337+0000","last_peered":"2024-09-11T05:23:13.315337+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:13.315337+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1949696,"num_objects":476,"num_object_clones":0,"num_object_copies":952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16891,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":908,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19438","reported_seq":19573,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:13.193245+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:13.193245+0000","last_peered":"2024-09-11T05:23:13.193245+0000","last_clean":"2024-09-11T05:23:13.193245+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:13.193245+0000","last_undegraded":"2024-09-11T05:23:13.193245+0000","last_fullsized":"2024-09-11T05:23:13.193245+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1826816,"num_objects":446,"num_object_clones":0,"num_object_copies":892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16741,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19583","reported_seq":19724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:12.800891+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:12.800891+0000","last_peered":"2024-09-11T05:23:12.800891+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:12.800891+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589248,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17651,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19241","reported_seq":19415,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.269683+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:13.269683+0000","last_peered":"2024-09-11T05:23:13.269683+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:13.269683+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1765376,"num_objects":431,"num_object_clones":0,"num_object_copies":862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15994,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19607","reported_seq":19735,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:12.768268+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:12.768268+0000","last_peered":"2024-09-11T05:23:12.768268+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:12.768268+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17636,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3096}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19265","reported_seq":26359,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.803289+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:12.803289+0000","last_peered":"2024-09-11T05:23:12.803289+0000","last_clean":"2024-09-11T05:23:12.803289+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:12.803289+0000","last_undegraded":"2024-09-11T05:23:12.803289+0000","last_fullsized":"2024-09-11T05:23:12.803289+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814528,"num_objects":443,"num_object_clones":0,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17168,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19718","reported_seq":19860,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.321658+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:13.321658+0000","last_peered":"2024-09-11T05:23:13.321658+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:13.321658+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16751,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19210","reported_seq":19354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.301508+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:13.301508+0000","last_peered":"2024-09-11T05:23:13.301508+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:13.301508+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818624,"num_objects":444,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19342","reported_seq":19488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.265039+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:13.265039+0000","last_peered":"2024-09-11T05:23:13.265039+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:13.265039+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1875968,"num_objects":458,"num_object_clones":0,"num_object_copies":916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16100,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19202","reported_seq":19356,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.232590+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:13.232590+0000","last_peered":"2024-09-11T05:23:13.232590+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:13.232590+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589865,"num_objects":389,"num_object_clones":0,"num_object_copies":778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16209,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":28959361,"num_objects":7072,"num_object_clones":0,"num_object_copies":14144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274623,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11486,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233213952,"data_stored":233143186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160559,"ondisk_log_size":160559,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461368,"kb_used_data":128424,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93910472,"statfs":{"total":96636764160,"available":96164323328,"internally_reserved":0,"allocated":131506176,"data_stored":131226310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":340913835},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466468,"kb_used_data":23824,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93905372,"statfs":{"total":96636764160,"available":96159100928,"internally_reserved":0,"allocated":24395776,"data_stored":24059648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453242539},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215232,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533276,"kb_used_data":78208,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93838564,"statfs":{"total":96636764160,"available":96090689536,"internally_reserved":0,"allocated":80084992,"data_stored":79805121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,4,2,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":78868480,"data_stored":78865001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23429120,"data_stored":23369384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130916352,"data_stored":130908801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:17.244 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:18.048 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:18.049 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:18.103 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-11T05:23:17.644878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27681428,"num_objects":6762,"num_object_clones":0,"num_object_copies":13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274937,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11487,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160674,"ondisk_log_size":160674,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462436,"kb_used_data":223460,"kb_used_omap":12,"kb_used_meta":1238899,"kb_avail":281653084,"statfs":{"total":289910292480,"available":288412758016,"internally_reserved":0,"allocated":228823040,"data_stored":227927175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1268632577},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,10,10,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19857408,"num_objects":-4848,"num_object_clones":0,"num_object_copies":-9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4848,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006167"},"pg_stats":[{"pgid":"2.f","version":"49'19752","reported_seq":19840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.320949+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:13.320949+0000","last_peered":"2024-09-11T05:23:13.320949+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:13.320949+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1794048,"num_objects":438,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3237}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19482","reported_seq":19595,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.809467+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:12.809467+0000","last_peered":"2024-09-11T05:23:12.809467+0000","last_clean":"2024-09-11T05:23:12.809467+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:12.809467+0000","last_undegraded":"2024-09-11T05:23:12.809467+0000","last_fullsized":"2024-09-11T05:23:12.809467+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712128,"num_objects":418,"num_object_clones":0,"num_object_copies":836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17880,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19393","reported_seq":26529,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:16.801120+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:16.801120+0000","last_peered":"2024-09-11T05:23:16.801120+0000","last_clean":"2024-09-11T05:23:16.801120+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:16.801120+0000","last_undegraded":"2024-09-11T05:23:16.801120+0000","last_fullsized":"2024-09-11T05:23:16.801120+0000","mapping_epoch":48,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1327128,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17796,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19505","reported_seq":19587,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.822494+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:12.822494+0000","last_peered":"2024-09-11T05:23:12.822494+0000","last_clean":"2024-09-11T05:23:12.822494+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:12.822494+0000","last_undegraded":"2024-09-11T05:23:12.822494+0000","last_fullsized":"2024-09-11T05:23:12.822494+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1863680,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19505,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19729","reported_seq":19869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:13.286428+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:13.286428+0000","last_peered":"2024-09-11T05:23:13.286428+0000","last_clean":"2024-09-11T05:23:13.286428+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:13.286428+0000","last_undegraded":"2024-09-11T05:23:13.286428+0000","last_fullsized":"2024-09-11T05:23:13.286428+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1802240,"num_objects":440,"num_object_clones":0,"num_object_copies":880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16563,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19678","reported_seq":27034,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:16.732613+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:16.732613+0000","last_peered":"2024-09-11T05:23:16.732613+0000","last_clean":"2024-09-11T05:23:16.732613+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:16.732613+0000","last_undegraded":"2024-09-11T05:23:16.732613+0000","last_fullsized":"2024-09-11T05:23:16.732613+0000","mapping_epoch":46,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462272,"num_objects":357,"num_object_clones":0,"num_object_copies":714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16222,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19726","reported_seq":21686,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.315337+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:13.315337+0000","last_peered":"2024-09-11T05:23:13.315337+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:13.315337+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1949696,"num_objects":476,"num_object_clones":0,"num_object_copies":952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16891,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":908,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19438","reported_seq":19573,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:13.193245+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:13.193245+0000","last_peered":"2024-09-11T05:23:13.193245+0000","last_clean":"2024-09-11T05:23:13.193245+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:13.193245+0000","last_undegraded":"2024-09-11T05:23:13.193245+0000","last_fullsized":"2024-09-11T05:23:13.193245+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1826816,"num_objects":446,"num_object_clones":0,"num_object_copies":892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16741,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19583","reported_seq":19724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:12.800891+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:12.800891+0000","last_peered":"2024-09-11T05:23:12.800891+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:12.800891+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589248,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17651,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19241","reported_seq":19415,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.269683+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:13.269683+0000","last_peered":"2024-09-11T05:23:13.269683+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:13.269683+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1765376,"num_objects":431,"num_object_clones":0,"num_object_copies":862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15994,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19607","reported_seq":19735,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:12.768268+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:12.768268+0000","last_peered":"2024-09-11T05:23:12.768268+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:12.768268+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17636,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3096}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19265","reported_seq":26359,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.803289+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:12.803289+0000","last_peered":"2024-09-11T05:23:12.803289+0000","last_clean":"2024-09-11T05:23:12.803289+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:12.803289+0000","last_undegraded":"2024-09-11T05:23:12.803289+0000","last_fullsized":"2024-09-11T05:23:12.803289+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814528,"num_objects":443,"num_object_clones":0,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17168,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19718","reported_seq":19860,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.321658+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:13.321658+0000","last_peered":"2024-09-11T05:23:13.321658+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:13.321658+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16751,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19210","reported_seq":19354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.301508+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:13.301508+0000","last_peered":"2024-09-11T05:23:13.301508+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:13.301508+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818624,"num_objects":444,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19342","reported_seq":19488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.265039+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:13.265039+0000","last_peered":"2024-09-11T05:23:13.265039+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:13.265039+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1875968,"num_objects":458,"num_object_clones":0,"num_object_copies":916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16100,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19202","reported_seq":19356,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.232590+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:13.232590+0000","last_peered":"2024-09-11T05:23:13.232590+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:13.232590+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589865,"num_objects":389,"num_object_clones":0,"num_object_copies":778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16209,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":27681409,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274935,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11486,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225984512,"data_stored":225913746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160671,"ondisk_log_size":160671,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461368,"kb_used_data":128424,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93910472,"statfs":{"total":96636764160,"available":96164323328,"internally_reserved":0,"allocated":131506176,"data_stored":131226310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":340913835},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467792,"kb_used_data":16828,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93904048,"statfs":{"total":96636764160,"available":96157745152,"internally_reserved":0,"allocated":17231872,"data_stored":16895744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215232,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533276,"kb_used_data":78208,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93838564,"statfs":{"total":96636764160,"available":96090689536,"internally_reserved":0,"allocated":80084992,"data_stored":79805121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,4,2,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":78868480,"data_stored":78865001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16199680,"data_stored":16139944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130916352,"data_stored":130908801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:18.104 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:18.801 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:18.804 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:18.881 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-11T05:23:17.644878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27681428,"num_objects":6762,"num_object_clones":0,"num_object_copies":13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274937,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11487,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160674,"ondisk_log_size":160674,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462436,"kb_used_data":223460,"kb_used_omap":12,"kb_used_meta":1238899,"kb_avail":281653084,"statfs":{"total":289910292480,"available":288412758016,"internally_reserved":0,"allocated":228823040,"data_stored":227927175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1268632577},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,10,10,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19857408,"num_objects":-4848,"num_object_clones":0,"num_object_copies":-9696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4848,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006167"},"pg_stats":[{"pgid":"2.f","version":"49'19752","reported_seq":19840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.320949+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:13.320949+0000","last_peered":"2024-09-11T05:23:13.320949+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:13.320949+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1794048,"num_objects":438,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3237}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19482","reported_seq":19595,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.809467+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:12.809467+0000","last_peered":"2024-09-11T05:23:12.809467+0000","last_clean":"2024-09-11T05:23:12.809467+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:12.809467+0000","last_undegraded":"2024-09-11T05:23:12.809467+0000","last_fullsized":"2024-09-11T05:23:12.809467+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712128,"num_objects":418,"num_object_clones":0,"num_object_copies":836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17880,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19393","reported_seq":26529,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:16.801120+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:16.801120+0000","last_peered":"2024-09-11T05:23:16.801120+0000","last_clean":"2024-09-11T05:23:16.801120+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:16.801120+0000","last_undegraded":"2024-09-11T05:23:16.801120+0000","last_fullsized":"2024-09-11T05:23:16.801120+0000","mapping_epoch":48,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1327128,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17796,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19505","reported_seq":19587,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.822494+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:12.822494+0000","last_peered":"2024-09-11T05:23:12.822494+0000","last_clean":"2024-09-11T05:23:12.822494+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:12.822494+0000","last_undegraded":"2024-09-11T05:23:12.822494+0000","last_fullsized":"2024-09-11T05:23:12.822494+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1863680,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19505,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19729","reported_seq":19869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:13.286428+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:13.286428+0000","last_peered":"2024-09-11T05:23:13.286428+0000","last_clean":"2024-09-11T05:23:13.286428+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:13.286428+0000","last_undegraded":"2024-09-11T05:23:13.286428+0000","last_fullsized":"2024-09-11T05:23:13.286428+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1802240,"num_objects":440,"num_object_clones":0,"num_object_copies":880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16563,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19678","reported_seq":27034,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:16.732613+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:16.732613+0000","last_peered":"2024-09-11T05:23:16.732613+0000","last_clean":"2024-09-11T05:23:16.732613+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:16.732613+0000","last_undegraded":"2024-09-11T05:23:16.732613+0000","last_fullsized":"2024-09-11T05:23:16.732613+0000","mapping_epoch":46,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462272,"num_objects":357,"num_object_clones":0,"num_object_copies":714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16222,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19726","reported_seq":21686,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.315337+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:13.315337+0000","last_peered":"2024-09-11T05:23:13.315337+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:13.315337+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1949696,"num_objects":476,"num_object_clones":0,"num_object_copies":952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16891,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":908,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19438","reported_seq":19573,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:13.193245+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:13.193245+0000","last_peered":"2024-09-11T05:23:13.193245+0000","last_clean":"2024-09-11T05:23:13.193245+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:13.193245+0000","last_undegraded":"2024-09-11T05:23:13.193245+0000","last_fullsized":"2024-09-11T05:23:13.193245+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1826816,"num_objects":446,"num_object_clones":0,"num_object_copies":892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16741,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19583","reported_seq":19724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:12.800891+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:12.800891+0000","last_peered":"2024-09-11T05:23:12.800891+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:12.800891+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589248,"num_objects":388,"num_object_clones":0,"num_object_copies":776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17651,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19241","reported_seq":19415,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.269683+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:13.269683+0000","last_peered":"2024-09-11T05:23:13.269683+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:13.269683+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1765376,"num_objects":431,"num_object_clones":0,"num_object_copies":862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15994,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19607","reported_seq":19735,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:12.768268+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:12.768268+0000","last_peered":"2024-09-11T05:23:12.768268+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:12.768268+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17636,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3096}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19265","reported_seq":26359,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:12.803289+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:12.803289+0000","last_peered":"2024-09-11T05:23:12.803289+0000","last_clean":"2024-09-11T05:23:12.803289+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:12.803289+0000","last_undegraded":"2024-09-11T05:23:12.803289+0000","last_fullsized":"2024-09-11T05:23:12.803289+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814528,"num_objects":443,"num_object_clones":0,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17168,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19718","reported_seq":19860,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.321658+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:13.321658+0000","last_peered":"2024-09-11T05:23:13.321658+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:13.321658+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16751,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3197}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19210","reported_seq":19354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.301508+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:13.301508+0000","last_peered":"2024-09-11T05:23:13.301508+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:13.301508+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818624,"num_objects":444,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19342","reported_seq":19488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.265039+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:13.265039+0000","last_peered":"2024-09-11T05:23:13.265039+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:13.265039+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1875968,"num_objects":458,"num_object_clones":0,"num_object_copies":916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16100,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19202","reported_seq":19356,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:13.232590+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:13.232590+0000","last_peered":"2024-09-11T05:23:13.232590+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:13.232590+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1589865,"num_objects":389,"num_object_clones":0,"num_object_copies":778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16209,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":27681409,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274935,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11486,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225984512,"data_stored":225913746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160671,"ondisk_log_size":160671,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461368,"kb_used_data":128424,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93910472,"statfs":{"total":96636764160,"available":96164323328,"internally_reserved":0,"allocated":131506176,"data_stored":131226310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":340913835},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467792,"kb_used_data":16828,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93904048,"statfs":{"total":96636764160,"available":96157745152,"internally_reserved":0,"allocated":17231872,"data_stored":16895744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215232,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533276,"kb_used_data":78208,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93838564,"statfs":{"total":96636764160,"available":96090689536,"internally_reserved":0,"allocated":80084992,"data_stored":79805121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,4,2,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":78868480,"data_stored":78865001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16199680,"data_stored":16139944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130916352,"data_stored":130908801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:18.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:18.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19752", 'reported_seq': 19840, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:13.320949+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:13.320949+0000', 'last_peered': '2024-09-11T05:23:13.320949+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:13.320949+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1794048, 'num_objects': 438, 'num_object_clones': 0, 'num_object_copies': 876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 438, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19752, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3237}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19726", 'reported_seq': 21686, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:13.315337+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:13.315337+0000', 'last_peered': '2024-09-11T05:23:13.315337+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:13.315337+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1949696, 'num_objects': 476, 'num_object_clones': 0, 'num_object_copies': 952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2670, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16891, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 908, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2194}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19583", 'reported_seq': 19724, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:12.800891+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:12.800891+0000', 'last_peered': '2024-09-11T05:23:12.800891+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:12.800891+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1589248, 'num_objects': 388, 'num_object_clones': 0, 'num_object_copies': 776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17651, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19241", 'reported_seq': 19415, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:13.269683+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:13.269683+0000', 'last_peered': '2024-09-11T05:23:13.269683+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:13.269683+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1765376, 'num_objects': 431, 'num_object_clones': 0, 'num_object_copies': 862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15994, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3011}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19607", 'reported_seq': 19735, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:12.768268+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:12.768268+0000', 'last_peered': '2024-09-11T05:23:12.768268+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:12.768268+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1744896, 'num_objects': 426, 'num_object_clones': 0, 'num_object_copies': 852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17636, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3096}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:18.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19718", 'reported_seq': 19860, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:13.321658+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:13.321658+0000', 'last_peered': '2024-09-11T05:23:13.321658+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:13.321658+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1744896, 'num_objects': 426, 'num_object_clones': 0, 'num_object_copies': 852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16751, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3197}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:18.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:18.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19210", 'reported_seq': 19354, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:13.301508+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:13.301508+0000', 'last_peered': '2024-09-11T05:23:13.301508+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:13.301508+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1818624, 'num_objects': 444, 'num_object_clones': 0, 'num_object_copies': 888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16076, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3051}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:18.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:18.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19342", 'reported_seq': 19488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:13.265039+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:13.265039+0000', 'last_peered': '2024-09-11T05:23:13.265039+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:13.265039+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1875968, 'num_objects': 458, 'num_object_clones': 0, 'num_object_copies': 916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16100, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:18.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:18.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19202", 'reported_seq': 19356, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:13.232590+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:13.232590+0000', 'last_peered': '2024-09-11T05:23:13.232590+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:13.232590+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1589865, 'num_objects': 389, 'num_object_clones': 0, 'num_object_copies': 778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16209, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:18.885 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:19.939 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:19.939 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30945/9470 objects degraded (326.769%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":677,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":4735,"data_bytes":19378836,"bytes_used":1460056064,"bytes_avail":288450236416,"bytes_total":289910292480,"degraded_objects":30945,"degraded_total":9470,"degraded_ratio":3.2676874340021116,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":549},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:19.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:19.997 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:20.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:20.859 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:20.913 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-11T05:23:19.648387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19378836,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276964,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11568,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1425836,"kb_used_data":211820,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281689684,"statfs":{"total":289910292480,"available":288450236416,"internally_reserved":0,"allocated":216903680,"data_stored":216007815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1243073537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,9,5,11,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27029504,"num_objects":-6599,"num_object_clones":0,"num_object_copies":-13198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6599,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005720"},"pg_stats":[{"pgid":"2.f","version":"49'19879","reported_seq":19968,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:18.328346+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:18.328346+0000","last_peered":"2024-09-11T05:23:18.328346+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:18.328346+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1273856,"num_objects":311,"num_object_clones":0,"num_object_copies":622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19879,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3364}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19627","reported_seq":19739,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:17.942485+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:17.942485+0000","last_peered":"2024-09-11T05:23:17.942485+0000","last_clean":"2024-09-11T05:23:17.942485+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:17.942485+0000","last_undegraded":"2024-09-11T05:23:17.942485+0000","last_fullsized":"2024-09-11T05:23:17.942485+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1118208,"num_objects":273,"num_object_clones":0,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18025,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19393","reported_seq":26529,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:16.801120+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:16.801120+0000","last_peered":"2024-09-11T05:23:16.801120+0000","last_clean":"2024-09-11T05:23:16.801120+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:16.801120+0000","last_undegraded":"2024-09-11T05:23:16.801120+0000","last_fullsized":"2024-09-11T05:23:16.801120+0000","mapping_epoch":48,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1327128,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17796,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19654","reported_seq":19736,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:17.942100+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:17.942100+0000","last_peered":"2024-09-11T05:23:17.942100+0000","last_clean":"2024-09-11T05:23:17.942100+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:17.942100+0000","last_undegraded":"2024-09-11T05:23:17.942100+0000","last_fullsized":"2024-09-11T05:23:17.942100+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1253376,"num_objects":306,"num_object_clones":0,"num_object_copies":612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19654,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19866","reported_seq":20006,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:18.313967+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:18.313967+0000","last_peered":"2024-09-11T05:23:18.313967+0000","last_clean":"2024-09-11T05:23:18.313967+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:18.313967+0000","last_undegraded":"2024-09-11T05:23:18.313967+0000","last_fullsized":"2024-09-11T05:23:18.313967+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1241088,"num_objects":303,"num_object_clones":0,"num_object_copies":606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16700,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19678","reported_seq":27034,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:16.732613+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:16.732613+0000","last_peered":"2024-09-11T05:23:16.732613+0000","last_clean":"2024-09-11T05:23:16.732613+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:16.732613+0000","last_undegraded":"2024-09-11T05:23:16.732613+0000","last_fullsized":"2024-09-11T05:23:16.732613+0000","mapping_epoch":46,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462272,"num_objects":357,"num_object_clones":0,"num_object_copies":714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16222,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19877","reported_seq":21999,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:18.341300+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:18.341300+0000","last_peered":"2024-09-11T05:23:18.341300+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:18.341300+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331200,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2589,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":989,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19577","reported_seq":19712,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:18.252658+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:18.252658+0000","last_peered":"2024-09-11T05:23:18.252658+0000","last_clean":"2024-09-11T05:23:18.252658+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:18.252658+0000","last_undegraded":"2024-09-11T05:23:18.252658+0000","last_fullsized":"2024-09-11T05:23:18.252658+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1257472,"num_objects":307,"num_object_clones":0,"num_object_copies":614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16880,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19716","reported_seq":19858,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:17.904939+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:17.904939+0000","last_peered":"2024-09-11T05:23:17.904939+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:17.904939+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044480,"num_objects":255,"num_object_clones":0,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17784,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19393","reported_seq":19569,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:18.281182+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:18.281182+0000","last_peered":"2024-09-11T05:23:18.281182+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:18.281182+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1142784,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16146,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3163}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.759726+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.759726+0000","last_peered":"2024-09-11T05:22:17.759726+0000","last_clean":"2024-09-11T05:22:17.759726+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.759726+0000","last_undegraded":"2024-09-11T05:22:17.759726+0000","last_fullsized":"2024-09-11T05:22:17.759726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19746","reported_seq":19874,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:17.901180+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:17.901180+0000","last_peered":"2024-09-11T05:23:17.901180+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:17.901180+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175552,"num_objects":287,"num_object_clones":0,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17775,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3235}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19424","reported_seq":26520,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:17.900645+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:17.900645+0000","last_peered":"2024-09-11T05:23:17.900645+0000","last_clean":"2024-09-11T05:23:17.900645+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:17.900645+0000","last_undegraded":"2024-09-11T05:23:17.900645+0000","last_fullsized":"2024-09-11T05:23:17.900645+0000","mapping_epoch":44,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1163264,"num_objects":284,"num_object_clones":0,"num_object_copies":568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17327,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19851","reported_seq":19993,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:18.143747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:18.143747+0000","last_peered":"2024-09-11T05:23:18.143747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:18.143747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1200128,"num_objects":293,"num_object_clones":0,"num_object_copies":586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16884,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19375","reported_seq":19517,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:18.288568+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:18.288568+0000","last_peered":"2024-09-11T05:23:18.288568+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:18.288568+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1142784,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16241,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19516","reported_seq":19662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:18.279761+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:18.279761+0000","last_peered":"2024-09-11T05:23:18.279761+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:18.279761+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1163264,"num_objects":284,"num_object_clones":0,"num_object_copies":568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16274,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19326","reported_seq":19480,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:18.313020+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:18.313020+0000","last_peered":"2024-09-11T05:23:18.313020+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:18.313020+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081961,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16333,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":19378817,"num_objects":4733,"num_object_clones":0,"num_object_copies":9466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4733,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276962,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11567,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":214097920,"data_stored":214027154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444224,"kb_used_data":123760,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93927616,"statfs":{"total":96636764160,"available":96181878784,"internally_reserved":0,"allocated":126730240,"data_stored":126450374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328134315},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467792,"kb_used_data":16828,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93904048,"statfs":{"total":96636764160,"available":96157745152,"internally_reserved":0,"allocated":17231872,"data_stored":16895744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215233,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513820,"kb_used_data":71232,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93858020,"statfs":{"total":96636764160,"available":96110612480,"internally_reserved":0,"allocated":72941568,"data_stored":72661697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453177003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,1,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":71761920,"data_stored":71758441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16199680,"data_stored":16139944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126136320,"data_stored":126128769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:23.915 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:24.805 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:24.807 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:24.864 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-11T05:23:23.649349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10052244,"num_objects":2458,"num_object_clones":0,"num_object_copies":4916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279241,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11652,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160578,"ondisk_log_size":160578,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442484,"kb_used_data":193772,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281673036,"statfs":{"total":289910292480,"available":288433188864,"internally_reserved":0,"allocated":198422528,"data_stored":197526663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1278594049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,7,6,7,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28356608,"num_objects":-6923,"num_object_clones":0,"num_object_copies":-13846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6923,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005858"},"pg_stats":[{"pgid":"2.f","version":"49'20021","reported_seq":20110,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.358339+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:23.358339+0000","last_peered":"2024-09-11T05:23:23.358339+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:23.358339+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20021,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3506}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19758","reported_seq":19871,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.883668+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:22.883668+0000","last_peered":"2024-09-11T05:23:22.883668+0000","last_clean":"2024-09-11T05:23:22.883668+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:22.883668+0000","last_undegraded":"2024-09-11T05:23:22.883668+0000","last_fullsized":"2024-09-11T05:23:22.883668+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":581632,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18156,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19534","reported_seq":26671,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:21.843347+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:21.843347+0000","last_peered":"2024-09-11T05:23:21.843347+0000","last_clean":"2024-09-11T05:23:21.843347+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:21.843347+0000","last_undegraded":"2024-09-11T05:23:21.843347+0000","last_fullsized":"2024-09-11T05:23:21.843347+0000","mapping_epoch":48,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749592,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17937,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19809","reported_seq":19891,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.859546+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:22.859546+0000","last_peered":"2024-09-11T05:23:22.859546+0000","last_clean":"2024-09-11T05:23:22.859546+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:22.859546+0000","last_undegraded":"2024-09-11T05:23:22.859546+0000","last_fullsized":"2024-09-11T05:23:22.859546+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19809,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20149,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:23.328533+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:23.328533+0000","last_peered":"2024-09-11T05:23:23.328533+0000","last_clean":"2024-09-11T05:23:23.328533+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:23.328533+0000","last_undegraded":"2024-09-11T05:23:23.328533+0000","last_fullsized":"2024-09-11T05:23:23.328533+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16843,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19845","reported_seq":27201,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:21.819439+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:21.819439+0000","last_peered":"2024-09-11T05:23:21.819439+0000","last_clean":"2024-09-11T05:23:21.819439+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:21.819439+0000","last_undegraded":"2024-09-11T05:23:21.819439+0000","last_fullsized":"2024-09-11T05:23:21.819439+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":778240,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16389,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20029","reported_seq":22319,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.298360+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:23.298360+0000","last_peered":"2024-09-11T05:23:23.298360+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:23.298360+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":708608,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17194,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1073,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19715","reported_seq":19850,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:23.345287+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:23.345287+0000","last_peered":"2024-09-11T05:23:23.345287+0000","last_clean":"2024-09-11T05:23:23.345287+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:23.345287+0000","last_undegraded":"2024-09-11T05:23:23.345287+0000","last_fullsized":"2024-09-11T05:23:23.345287+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17018,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19833","reported_seq":19975,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:22.841601+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:22.841601+0000","last_peered":"2024-09-11T05:23:22.841601+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:22.841601+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":565248,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17901,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3387}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19542","reported_seq":19718,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.381626+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:23.381626+0000","last_peered":"2024-09-11T05:23:23.381626+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:23.381626+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":532480,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16295,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19880","reported_seq":20004,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:22.907040+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:22.907040+0000","last_peered":"2024-09-11T05:23:22.907040+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:22.907040+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":626688,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19554","reported_seq":26650,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.855280+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:22.855280+0000","last_peered":"2024-09-11T05:23:22.855280+0000","last_clean":"2024-09-11T05:23:22.855280+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:22.855280+0000","last_undegraded":"2024-09-11T05:23:22.855280+0000","last_fullsized":"2024-09-11T05:23:22.855280+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":630784,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17457,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20003","reported_seq":20145,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.333346+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:23.333346+0000","last_peered":"2024-09-11T05:23:23.333346+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:23.333346+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577536,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17036,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19515","reported_seq":19659,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.303259+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:23.303259+0000","last_peered":"2024-09-11T05:23:23.303259+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:23.303259+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16381,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19663","reported_seq":19809,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.269786+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:23.269786+0000","last_peered":"2024-09-11T05:23:23.269786+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:23.269786+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16421,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19465","reported_seq":19619,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.379867+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:23.379867+0000","last_peered":"2024-09-11T05:23:23.379867+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:23.379867+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":512617,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":10052225,"num_objects":2456,"num_object_clones":0,"num_object_copies":4912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279239,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11651,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195416064,"data_stored":195345298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160575,"ondisk_log_size":160575,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462112,"kb_used_data":119432,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93909728,"statfs":{"total":96636764160,"available":96163561472,"internally_reserved":0,"allocated":122298368,"data_stored":122018502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465220,"kb_used_data":10092,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93906620,"statfs":{"total":96636764160,"available":96160378880,"internally_reserved":0,"allocated":10334208,"data_stored":9998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515152,"kb_used_data":64248,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93856688,"statfs":{"total":96636764160,"available":96109248512,"internally_reserved":0,"allocated":65789952,"data_stored":65510081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64533097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9203712,"data_stored":9143976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121675776,"data_stored":121668225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:24.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:25.812 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:25.902 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-11T05:23:25.649829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10052244,"num_objects":2458,"num_object_clones":0,"num_object_copies":4916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279241,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11652,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160578,"ondisk_log_size":160578,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442484,"kb_used_data":193772,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281673036,"statfs":{"total":289910292480,"available":288433188864,"internally_reserved":0,"allocated":198422528,"data_stored":197526663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1278594049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,7,6,7,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18907136,"num_objects":-4616,"num_object_clones":0,"num_object_copies":-9232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4616,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005926"},"pg_stats":[{"pgid":"2.f","version":"49'20021","reported_seq":20110,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.358339+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:23.358339+0000","last_peered":"2024-09-11T05:23:23.358339+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:23.358339+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20021,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3506}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19758","reported_seq":19871,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.883668+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:22.883668+0000","last_peered":"2024-09-11T05:23:22.883668+0000","last_clean":"2024-09-11T05:23:22.883668+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:22.883668+0000","last_undegraded":"2024-09-11T05:23:22.883668+0000","last_fullsized":"2024-09-11T05:23:22.883668+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":581632,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18156,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19534","reported_seq":26671,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:21.843347+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:21.843347+0000","last_peered":"2024-09-11T05:23:21.843347+0000","last_clean":"2024-09-11T05:23:21.843347+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:21.843347+0000","last_undegraded":"2024-09-11T05:23:21.843347+0000","last_fullsized":"2024-09-11T05:23:21.843347+0000","mapping_epoch":48,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749592,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17937,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19809","reported_seq":19891,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.859546+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:22.859546+0000","last_peered":"2024-09-11T05:23:22.859546+0000","last_clean":"2024-09-11T05:23:22.859546+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:22.859546+0000","last_undegraded":"2024-09-11T05:23:22.859546+0000","last_fullsized":"2024-09-11T05:23:22.859546+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19809,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20149,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:23.328533+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:23.328533+0000","last_peered":"2024-09-11T05:23:23.328533+0000","last_clean":"2024-09-11T05:23:23.328533+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:23.328533+0000","last_undegraded":"2024-09-11T05:23:23.328533+0000","last_fullsized":"2024-09-11T05:23:23.328533+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16843,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19845","reported_seq":27201,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:21.819439+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:21.819439+0000","last_peered":"2024-09-11T05:23:21.819439+0000","last_clean":"2024-09-11T05:23:21.819439+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:21.819439+0000","last_undegraded":"2024-09-11T05:23:21.819439+0000","last_fullsized":"2024-09-11T05:23:21.819439+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":778240,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16389,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20029","reported_seq":22319,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.298360+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:23.298360+0000","last_peered":"2024-09-11T05:23:23.298360+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:23.298360+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":708608,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17194,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1073,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19715","reported_seq":19850,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:23.345287+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:23.345287+0000","last_peered":"2024-09-11T05:23:23.345287+0000","last_clean":"2024-09-11T05:23:23.345287+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:23.345287+0000","last_undegraded":"2024-09-11T05:23:23.345287+0000","last_fullsized":"2024-09-11T05:23:23.345287+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17018,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19833","reported_seq":19975,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:22.841601+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:22.841601+0000","last_peered":"2024-09-11T05:23:22.841601+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:22.841601+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":565248,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17901,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3387}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19542","reported_seq":19718,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.381626+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:23.381626+0000","last_peered":"2024-09-11T05:23:23.381626+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:23.381626+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":532480,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16295,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19880","reported_seq":20004,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:22.907040+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:22.907040+0000","last_peered":"2024-09-11T05:23:22.907040+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:22.907040+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":626688,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19554","reported_seq":26650,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.855280+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:22.855280+0000","last_peered":"2024-09-11T05:23:22.855280+0000","last_clean":"2024-09-11T05:23:22.855280+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:22.855280+0000","last_undegraded":"2024-09-11T05:23:22.855280+0000","last_fullsized":"2024-09-11T05:23:22.855280+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":630784,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17457,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20003","reported_seq":20145,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.333346+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:23.333346+0000","last_peered":"2024-09-11T05:23:23.333346+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:23.333346+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577536,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17036,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19515","reported_seq":19659,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.303259+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:23.303259+0000","last_peered":"2024-09-11T05:23:23.303259+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:23.303259+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16381,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19663","reported_seq":19809,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.269786+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:23.269786+0000","last_peered":"2024-09-11T05:23:23.269786+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:23.269786+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16421,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19465","reported_seq":19619,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.379867+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:23.379867+0000","last_peered":"2024-09-11T05:23:23.379867+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:23.379867+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":512617,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":10052225,"num_objects":2456,"num_object_clones":0,"num_object_copies":4912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279239,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11651,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195416064,"data_stored":195345298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160575,"ondisk_log_size":160575,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462112,"kb_used_data":119432,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93909728,"statfs":{"total":96636764160,"available":96163561472,"internally_reserved":0,"allocated":122298368,"data_stored":122018502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465220,"kb_used_data":10092,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93906620,"statfs":{"total":96636764160,"available":96160378880,"internally_reserved":0,"allocated":10334208,"data_stored":9998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":466022059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515152,"kb_used_data":64248,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93856688,"statfs":{"total":96636764160,"available":96109248512,"internally_reserved":0,"allocated":65789952,"data_stored":65510081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64533097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9203712,"data_stored":9143976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121675776,"data_stored":121668225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20021", 'reported_seq': 20110, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:23.358339+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:23.358339+0000', 'last_peered': '2024-09-11T05:23:23.358339+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:23.358339+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 692224, 'num_objects': 169, 'num_object_clones': 0, 'num_object_copies': 338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20021, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3506}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'20029", 'reported_seq': 22319, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:23.298360+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:23.298360+0000', 'last_peered': '2024-09-11T05:23:23.298360+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:23.298360+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 708608, 'num_objects': 173, 'num_object_clones': 0, 'num_object_copies': 346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2505, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17194, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1073, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2332}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19833", 'reported_seq': 19975, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:22.841601+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:22.841601+0000', 'last_peered': '2024-09-11T05:23:22.841601+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:22.841601+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 565248, 'num_objects': 138, 'num_object_clones': 0, 'num_object_copies': 276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 138, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17901, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3387}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19542", 'reported_seq': 19718, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:23.381626+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:23.381626+0000', 'last_peered': '2024-09-11T05:23:23.381626+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:23.381626+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 532480, 'num_objects': 130, 'num_object_clones': 0, 'num_object_copies': 260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16295, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3312}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:25.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19880", 'reported_seq': 20004, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:22.907040+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:22.907040+0000', 'last_peered': '2024-09-11T05:23:22.907040+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:22.907040+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 626688, 'num_objects': 153, 'num_object_clones': 0, 'num_object_copies': 306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 153, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17909, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3369}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20003", 'reported_seq': 20145, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:23.333346+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:23.333346+0000', 'last_peered': '2024-09-11T05:23:23.333346+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:23.333346+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 577536, 'num_objects': 141, 'num_object_clones': 0, 'num_object_copies': 282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 141, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17036, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3482}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19515", 'reported_seq': 19659, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:23.303259+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:23.303259+0000', 'last_peered': '2024-09-11T05:23:23.303259+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:23.303259+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 569344, 'num_objects': 139, 'num_object_clones': 0, 'num_object_copies': 278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16381, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3356}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19663", 'reported_seq': 19809, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:23.269786+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:23.269786+0000', 'last_peered': '2024-09-11T05:23:23.269786+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:23.269786+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 561152, 'num_objects': 137, 'num_object_clones': 0, 'num_object_copies': 274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16421, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:25.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19465", 'reported_seq': 19619, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:23.379867+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:23.379867+0000', 'last_peered': '2024-09-11T05:23:23.379867+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:23.379867+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 512617, 'num_objects': 126, 'num_object_clones': 0, 'num_object_copies': 252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16472, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3389}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:25.906 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:27.104 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:27.104 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30861/4916 objects degraded (627.766%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":2458,"data_bytes":10052244,"bytes_used":1477103616,"bytes_avail":288433188864,"bytes_total":289910292480,"degraded_objects":30861,"degraded_total":4916,"degraded_ratio":6.2776647681041489,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:27.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:27.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:27.983 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:27.984 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:28.034 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-11T05:23:27.650330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8852116,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279534,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11652,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160671,"ondisk_log_size":160671,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1427284,"kb_used_data":186892,"kb_used_omap":12,"kb_used_meta":1240307,"kb_avail":281688236,"statfs":{"total":289910292480,"available":288448753664,"internally_reserved":0,"allocated":191377408,"data_stored":190481543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1270074369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,5,6,7,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20107264,"num_objects":-4909,"num_object_clones":0,"num_object_copies":-9818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4909,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005893"},"pg_stats":[{"pgid":"2.f","version":"49'20021","reported_seq":20110,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.358339+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:23.358339+0000","last_peered":"2024-09-11T05:23:23.358339+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:23.358339+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20021,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3506}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19758","reported_seq":19871,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.883668+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:22.883668+0000","last_peered":"2024-09-11T05:23:22.883668+0000","last_clean":"2024-09-11T05:23:22.883668+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:22.883668+0000","last_undegraded":"2024-09-11T05:23:22.883668+0000","last_fullsized":"2024-09-11T05:23:22.883668+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":581632,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18156,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19682","reported_seq":26819,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.948327+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:26.948327+0000","last_peered":"2024-09-11T05:23:26.948327+0000","last_clean":"2024-09-11T05:23:26.948327+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:26.948327+0000","last_undegraded":"2024-09-11T05:23:26.948327+0000","last_fullsized":"2024-09-11T05:23:26.948327+0000","mapping_epoch":48,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143384,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18085,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19809","reported_seq":19891,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.859546+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:22.859546+0000","last_peered":"2024-09-11T05:23:22.859546+0000","last_clean":"2024-09-11T05:23:22.859546+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:22.859546+0000","last_undegraded":"2024-09-11T05:23:22.859546+0000","last_fullsized":"2024-09-11T05:23:22.859546+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19809,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20149,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:23.328533+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:23.328533+0000","last_peered":"2024-09-11T05:23:23.328533+0000","last_clean":"2024-09-11T05:23:23.328533+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:23.328533+0000","last_undegraded":"2024-09-11T05:23:23.328533+0000","last_fullsized":"2024-09-11T05:23:23.328533+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16843,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19990","reported_seq":27346,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.714743+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:26.714743+0000","last_peered":"2024-09-11T05:23:26.714743+0000","last_clean":"2024-09-11T05:23:26.714743+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:26.714743+0000","last_undegraded":"2024-09-11T05:23:26.714743+0000","last_fullsized":"2024-09-11T05:23:26.714743+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184320,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16534,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20029","reported_seq":22319,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.298360+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:23.298360+0000","last_peered":"2024-09-11T05:23:23.298360+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:23.298360+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":708608,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17194,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1073,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19715","reported_seq":19850,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:23.345287+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:23.345287+0000","last_peered":"2024-09-11T05:23:23.345287+0000","last_clean":"2024-09-11T05:23:23.345287+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:23.345287+0000","last_undegraded":"2024-09-11T05:23:23.345287+0000","last_fullsized":"2024-09-11T05:23:23.345287+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17018,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19833","reported_seq":19975,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:22.841601+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:22.841601+0000","last_peered":"2024-09-11T05:23:22.841601+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:22.841601+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":565248,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17901,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3387}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19542","reported_seq":19718,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.381626+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:23.381626+0000","last_peered":"2024-09-11T05:23:23.381626+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:23.381626+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":532480,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16295,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19880","reported_seq":20004,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:22.907040+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:22.907040+0000","last_peered":"2024-09-11T05:23:22.907040+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:22.907040+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":626688,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17909,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19554","reported_seq":26650,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:22.855280+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:22.855280+0000","last_peered":"2024-09-11T05:23:22.855280+0000","last_clean":"2024-09-11T05:23:22.855280+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:22.855280+0000","last_undegraded":"2024-09-11T05:23:22.855280+0000","last_fullsized":"2024-09-11T05:23:22.855280+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":630784,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17457,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20003","reported_seq":20145,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.333346+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:23.333346+0000","last_peered":"2024-09-11T05:23:23.333346+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:23.333346+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577536,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17036,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19515","reported_seq":19659,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.303259+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:23.303259+0000","last_peered":"2024-09-11T05:23:23.303259+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:23.303259+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16381,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19663","reported_seq":19809,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.269786+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:23.269786+0000","last_peered":"2024-09-11T05:23:23.269786+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:23.269786+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16421,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19465","reported_seq":19619,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:23.379867+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:23.379867+0000","last_peered":"2024-09-11T05:23:23.379867+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:23.379867+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":512617,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":8852097,"num_objects":2163,"num_object_clones":0,"num_object_copies":4326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279532,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11651,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188317696,"data_stored":188246930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462112,"kb_used_data":119432,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93909728,"statfs":{"total":96636764160,"available":96163561472,"internally_reserved":0,"allocated":122298368,"data_stored":122018502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350875307},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450020,"kb_used_data":3212,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93921820,"statfs":{"total":96636764160,"available":96175943680,"internally_reserved":0,"allocated":3289088,"data_stored":2952960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515152,"kb_used_data":64248,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93856688,"statfs":{"total":96636764160,"available":96109248512,"internally_reserved":0,"allocated":65789952,"data_stored":65510081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64533097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2105344,"data_stored":2045608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121675776,"data_stored":121668225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:28.433 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Removed 159273 objects 2024-09-11T05:23:28.433 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi133.stdout:Clean up completed and total clean up time :355.252 2024-09-11T05:23:31.037 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:31.420 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:31.421 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:31.440 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-11T05:23:29.650872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639636,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281539,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11725,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161076,"ondisk_log_size":161076,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1424196,"kb_used_data":175492,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281691324,"statfs":{"total":289910292480,"available":288451915776,"internally_reserved":0,"allocated":179703808,"data_stored":178807943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1278594049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,4,13,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27041792,"num_objects":-6602,"num_object_clones":0,"num_object_copies":-13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005994"},"pg_stats":[{"pgid":"2.f","version":"49'20186","reported_seq":20275,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.310185+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.310185+0000","last_peered":"2024-09-11T05:23:28.310185+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.310185+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20186,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19894","reported_seq":20006,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.951762+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:27.951762+0000","last_peered":"2024-09-11T05:23:27.951762+0000","last_clean":"2024-09-11T05:23:27.951762+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:27.951762+0000","last_undegraded":"2024-09-11T05:23:27.951762+0000","last_fullsized":"2024-09-11T05:23:27.951762+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18292,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19682","reported_seq":26819,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.948327+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:26.948327+0000","last_peered":"2024-09-11T05:23:26.948327+0000","last_clean":"2024-09-11T05:23:26.948327+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:26.948327+0000","last_undegraded":"2024-09-11T05:23:26.948327+0000","last_fullsized":"2024-09-11T05:23:26.948327+0000","mapping_epoch":48,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143384,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18085,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19944","reported_seq":20026,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.838170+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:27.838170+0000","last_peered":"2024-09-11T05:23:27.838170+0000","last_clean":"2024-09-11T05:23:27.838170+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:27.838170+0000","last_undegraded":"2024-09-11T05:23:27.838170+0000","last_fullsized":"2024-09-11T05:23:27.838170+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19944,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19990","reported_seq":27346,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.714743+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:26.714743+0000","last_peered":"2024-09-11T05:23:26.714743+0000","last_clean":"2024-09-11T05:23:26.714743+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:26.714743+0000","last_undegraded":"2024-09-11T05:23:26.714743+0000","last_fullsized":"2024-09-11T05:23:26.714743+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184320,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16534,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22638,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.416480+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:28.416480+0000","last_peered":"2024-09-11T05:23:28.416480+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:28.416480+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19883","reported_seq":20018,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.337496+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.337496+0000","last_peered":"2024-09-11T05:23:28.337496+0000","last_clean":"2024-09-11T05:23:28.337496+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.337496+0000","last_undegraded":"2024-09-11T05:23:28.337496+0000","last_fullsized":"2024-09-11T05:23:28.337496+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17186,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19956","reported_seq":20097,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:27.835666+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:27.835666+0000","last_peered":"2024-09-11T05:23:27.835666+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:27.835666+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61440,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20016","reported_seq":20144,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:27.961598+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:27.961598+0000","last_peered":"2024-09-11T05:23:27.961598+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:27.961598+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69632,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18045,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3505}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19694","reported_seq":26790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.936461+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:27.936461+0000","last_peered":"2024-09-11T05:23:27.936461+0000","last_clean":"2024-09-11T05:23:27.936461+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:27.936461+0000","last_undegraded":"2024-09-11T05:23:27.936461+0000","last_fullsized":"2024-09-11T05:23:27.936461+0000","mapping_epoch":44,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57344,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17597,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20284,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.308932+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.308932+0000","last_peered":"2024-09-11T05:23:28.308932+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.308932+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19799","reported_seq":19945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406442+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.406442+0000","last_peered":"2024-09-11T05:23:28.406442+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.406442+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16557,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19588","reported_seq":19742,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.260576+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.260576+0000","last_peered":"2024-09-11T05:23:28.260576+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.260576+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8809,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16595,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":639617,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281537,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11724,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176553984,"data_stored":176483218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161073,"ondisk_log_size":161073,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461796,"kb_used_data":114960,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910044,"statfs":{"total":96636764160,"available":96163885056,"internally_reserved":0,"allocated":117719040,"data_stored":117439174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450020,"kb_used_data":3212,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93921820,"statfs":{"total":96636764160,"available":96175943680,"internally_reserved":0,"allocated":3289088,"data_stored":2952960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215235,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512380,"kb_used_data":57320,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93859460,"statfs":{"total":96636764160,"available":96112087040,"internally_reserved":0,"allocated":58695680,"data_stored":58415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2105344,"data_stored":2045608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117022720,"data_stored":117015169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:31.442 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:31.793 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:31.794 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:31.814 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-11T05:23:31.651579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639636,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281539,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11725,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161076,"ondisk_log_size":161076,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1424196,"kb_used_data":175492,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281691324,"statfs":{"total":289910292480,"available":288451915776,"internally_reserved":0,"allocated":179703808,"data_stored":178807943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1278594049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,4,13,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18739200,"num_objects":-4575,"num_object_clones":0,"num_object_copies":-9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4575,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003192"},"pg_stats":[{"pgid":"2.f","version":"49'20186","reported_seq":20275,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.310185+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.310185+0000","last_peered":"2024-09-11T05:23:28.310185+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.310185+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20186,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19894","reported_seq":20006,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.951762+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:27.951762+0000","last_peered":"2024-09-11T05:23:27.951762+0000","last_clean":"2024-09-11T05:23:27.951762+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:27.951762+0000","last_undegraded":"2024-09-11T05:23:27.951762+0000","last_fullsized":"2024-09-11T05:23:27.951762+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18292,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19682","reported_seq":26819,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.948327+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:26.948327+0000","last_peered":"2024-09-11T05:23:26.948327+0000","last_clean":"2024-09-11T05:23:26.948327+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:26.948327+0000","last_undegraded":"2024-09-11T05:23:26.948327+0000","last_fullsized":"2024-09-11T05:23:26.948327+0000","mapping_epoch":48,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143384,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18085,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19944","reported_seq":20026,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.838170+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:27.838170+0000","last_peered":"2024-09-11T05:23:27.838170+0000","last_clean":"2024-09-11T05:23:27.838170+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:27.838170+0000","last_undegraded":"2024-09-11T05:23:27.838170+0000","last_fullsized":"2024-09-11T05:23:27.838170+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19944,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19990","reported_seq":27346,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.714743+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:26.714743+0000","last_peered":"2024-09-11T05:23:26.714743+0000","last_clean":"2024-09-11T05:23:26.714743+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:26.714743+0000","last_undegraded":"2024-09-11T05:23:26.714743+0000","last_fullsized":"2024-09-11T05:23:26.714743+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184320,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16534,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22638,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.416480+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:28.416480+0000","last_peered":"2024-09-11T05:23:28.416480+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:28.416480+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19883","reported_seq":20018,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.337496+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.337496+0000","last_peered":"2024-09-11T05:23:28.337496+0000","last_clean":"2024-09-11T05:23:28.337496+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.337496+0000","last_undegraded":"2024-09-11T05:23:28.337496+0000","last_fullsized":"2024-09-11T05:23:28.337496+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17186,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19956","reported_seq":20097,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:27.835666+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:27.835666+0000","last_peered":"2024-09-11T05:23:27.835666+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:27.835666+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61440,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20016","reported_seq":20144,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:27.961598+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:27.961598+0000","last_peered":"2024-09-11T05:23:27.961598+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:27.961598+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69632,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18045,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3505}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19694","reported_seq":26790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.936461+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:27.936461+0000","last_peered":"2024-09-11T05:23:27.936461+0000","last_clean":"2024-09-11T05:23:27.936461+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:27.936461+0000","last_undegraded":"2024-09-11T05:23:27.936461+0000","last_fullsized":"2024-09-11T05:23:27.936461+0000","mapping_epoch":44,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57344,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17597,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20284,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.308932+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.308932+0000","last_peered":"2024-09-11T05:23:28.308932+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.308932+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19799","reported_seq":19945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406442+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.406442+0000","last_peered":"2024-09-11T05:23:28.406442+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.406442+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16557,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19588","reported_seq":19742,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.260576+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.260576+0000","last_peered":"2024-09-11T05:23:28.260576+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.260576+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8809,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16595,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":639617,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281537,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11724,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176553984,"data_stored":176483218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161073,"ondisk_log_size":161073,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461796,"kb_used_data":114960,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910044,"statfs":{"total":96636764160,"available":96163885056,"internally_reserved":0,"allocated":117719040,"data_stored":117439174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450020,"kb_used_data":3212,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93921820,"statfs":{"total":96636764160,"available":96175943680,"internally_reserved":0,"allocated":3289088,"data_stored":2952960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215235,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512380,"kb_used_data":57320,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93859460,"statfs":{"total":96636764160,"available":96112087040,"internally_reserved":0,"allocated":58695680,"data_stored":58415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2105344,"data_stored":2045608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117022720,"data_stored":117015169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:31.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:31.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20186", 'reported_seq': 20275, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.310185+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:28.310185+0000', 'last_peered': '2024-09-11T05:23:28.310185+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:28.310185+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16384, 'num_objects': 4, 'num_object_clones': 0, 'num_object_copies': 8, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20186, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3671}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:31.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:31.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'20202", 'reported_seq': 22638, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.416480+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:28.416480+0000', 'last_peered': '2024-09-11T05:23:28.416480+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:28.416480+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2432, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17367, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1146, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2432}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19956", 'reported_seq': 20097, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:27.835666+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:27.835666+0000', 'last_peered': '2024-09-11T05:23:27.835666+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:27.835666+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 61440, 'num_objects': 15, 'num_object_clones': 0, 'num_object_copies': 30, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 15, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19848, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.365158+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:28.365158+0000', 'last_peered': '2024-09-11T05:23:28.365158+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:28.365158+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20016", 'reported_seq': 20144, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:27.961598+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:27.961598+0000', 'last_peered': '2024-09-11T05:23:27.961598+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:27.961598+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 69632, 'num_objects': 17, 'num_object_clones': 0, 'num_object_copies': 34, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 17, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18045, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3505}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20284, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.308932+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:28.308932+0000', 'last_peered': '2024-09-11T05:23:28.308932+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:28.308932+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:31.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:31.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19798, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.331223+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:28.331223+0000', 'last_peered': '2024-09-11T05:23:28.331223+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:28.331223+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:31.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:31.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19799", 'reported_seq': 19945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.406442+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:28.406442+0000', 'last_peered': '2024-09-11T05:23:28.406442+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:28.406442+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4096, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16557, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:31.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:31.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19588", 'reported_seq': 19742, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.260576+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:28.260576+0000', 'last_peered': '2024-09-11T05:23:28.260576+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:28.260576+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8809, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16595, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:31.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:32.221 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:32.221 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30788/320 objects degraded (9621.250%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":689,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160,"data_bytes":639636,"bytes_used":1458376704,"bytes_avail":288451915776,"bytes_total":289910292480,"degraded_objects":30788,"degraded_total":320,"degraded_ratio":96.212500000000006,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":381},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:32.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:32.241 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:32.609 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:32.610 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:32.629 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-11T05:23:31.651579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639636,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281539,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11725,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161076,"ondisk_log_size":161076,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1424196,"kb_used_data":175492,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281691324,"statfs":{"total":289910292480,"available":288451915776,"internally_reserved":0,"allocated":179703808,"data_stored":178807943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1278594049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,4,13,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18739200,"num_objects":-4575,"num_object_clones":0,"num_object_copies":-9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4575,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003192"},"pg_stats":[{"pgid":"2.f","version":"49'20186","reported_seq":20275,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.310185+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.310185+0000","last_peered":"2024-09-11T05:23:28.310185+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.310185+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20186,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19894","reported_seq":20006,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.951762+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:27.951762+0000","last_peered":"2024-09-11T05:23:27.951762+0000","last_clean":"2024-09-11T05:23:27.951762+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:27.951762+0000","last_undegraded":"2024-09-11T05:23:27.951762+0000","last_fullsized":"2024-09-11T05:23:27.951762+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18292,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19682","reported_seq":26819,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.948327+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:26.948327+0000","last_peered":"2024-09-11T05:23:26.948327+0000","last_clean":"2024-09-11T05:23:26.948327+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:26.948327+0000","last_undegraded":"2024-09-11T05:23:26.948327+0000","last_fullsized":"2024-09-11T05:23:26.948327+0000","mapping_epoch":48,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143384,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18085,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19944","reported_seq":20026,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.838170+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:27.838170+0000","last_peered":"2024-09-11T05:23:27.838170+0000","last_clean":"2024-09-11T05:23:27.838170+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:27.838170+0000","last_undegraded":"2024-09-11T05:23:27.838170+0000","last_fullsized":"2024-09-11T05:23:27.838170+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19944,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19990","reported_seq":27346,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:26.714743+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:26.714743+0000","last_peered":"2024-09-11T05:23:26.714743+0000","last_clean":"2024-09-11T05:23:26.714743+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:26.714743+0000","last_undegraded":"2024-09-11T05:23:26.714743+0000","last_fullsized":"2024-09-11T05:23:26.714743+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184320,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16534,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22638,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.416480+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:28.416480+0000","last_peered":"2024-09-11T05:23:28.416480+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:28.416480+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2432}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19883","reported_seq":20018,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.337496+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.337496+0000","last_peered":"2024-09-11T05:23:28.337496+0000","last_clean":"2024-09-11T05:23:28.337496+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.337496+0000","last_undegraded":"2024-09-11T05:23:28.337496+0000","last_fullsized":"2024-09-11T05:23:28.337496+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17186,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19956","reported_seq":20097,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:27.835666+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:27.835666+0000","last_peered":"2024-09-11T05:23:27.835666+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:27.835666+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61440,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20016","reported_seq":20144,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:27.961598+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:27.961598+0000","last_peered":"2024-09-11T05:23:27.961598+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:27.961598+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69632,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18045,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3505}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19694","reported_seq":26790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:27.936461+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:27.936461+0000","last_peered":"2024-09-11T05:23:27.936461+0000","last_clean":"2024-09-11T05:23:27.936461+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:27.936461+0000","last_undegraded":"2024-09-11T05:23:27.936461+0000","last_fullsized":"2024-09-11T05:23:27.936461+0000","mapping_epoch":44,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57344,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17597,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20284,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.308932+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.308932+0000","last_peered":"2024-09-11T05:23:28.308932+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.308932+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19799","reported_seq":19945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406442+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.406442+0000","last_peered":"2024-09-11T05:23:28.406442+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.406442+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16557,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19588","reported_seq":19742,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.260576+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.260576+0000","last_peered":"2024-09-11T05:23:28.260576+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.260576+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8809,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16595,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":639617,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281537,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11724,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176553984,"data_stored":176483218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161073,"ondisk_log_size":161073,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461796,"kb_used_data":114960,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910044,"statfs":{"total":96636764160,"available":96163885056,"internally_reserved":0,"allocated":117719040,"data_stored":117439174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450020,"kb_used_data":3212,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93921820,"statfs":{"total":96636764160,"available":96175943680,"internally_reserved":0,"allocated":3289088,"data_stored":2952960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457502379},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215235,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512380,"kb_used_data":57320,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93859460,"statfs":{"total":96636764160,"available":96112087040,"internally_reserved":0,"allocated":58695680,"data_stored":58415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2105344,"data_stored":2045608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117022720,"data_stored":117015169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:35.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:35.968 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:35.969 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:35.987 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-11T05:23:35.652995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12094,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1424068,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281691452,"statfs":{"total":289910292480,"available":288452046848,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1282853889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10051608,"num_objects":-2455,"num_object_clones":0,"num_object_copies":-4910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003645"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":23376,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:33.426073+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:33.426073+0000","last_peered":"2024-09-11T05:23:33.426073+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:33.426073+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.406031+0000","last_peered":"2024-09-11T05:23:28.406031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.406031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12093,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451428,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920412,"statfs":{"total":96636764160,"available":96174501888,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215236,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:35.988 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:36.324 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:36.325 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:36.346 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-11T05:23:35.652995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12094,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1424068,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281691452,"statfs":{"total":289910292480,"available":288452046848,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1282853889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10051608,"num_objects":-2455,"num_object_clones":0,"num_object_copies":-4910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003645"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":23376,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:33.426073+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:33.426073+0000","last_peered":"2024-09-11T05:23:33.426073+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:33.426073+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.406031+0000","last_peered":"2024-09-11T05:23:28.406031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.406031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12093,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451428,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920412,"statfs":{"total":96636764160,"available":96174501888,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215236,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20279, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435999+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:28.435999+0000', 'last_peered': '2024-09-11T05:23:28.435999+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:28.435999+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'20202", 'reported_seq': 23376, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:33.426073+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:33.426073+0000', 'last_peered': '2024-09-11T05:23:33.426073+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:33.426073+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2063, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17367, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1515, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20113, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.393703+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:28.393703+0000', 'last_peered': '2024-09-11T05:23:28.393703+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:28.393703+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:36.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19848, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.365158+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:28.365158+0000', 'last_peered': '2024-09-11T05:23:28.365158+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:28.365158+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 20161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.406031+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:28.406031+0000', 'last_peered': '2024-09-11T05:23:28.406031+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:28.406031+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3522}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20286, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435945+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:28.435945+0000', 'last_peered': '2024-09-11T05:23:28.435945+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:28.435945+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19798, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.331223+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:28.331223+0000', 'last_peered': '2024-09-11T05:23:28.331223+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:28.331223+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19946, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407386+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:28.407386+0000', 'last_peered': '2024-09-11T05:23:28.407386+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:28.407386+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:36.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19745, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.433745+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:28.433745+0000', 'last_peered': '2024-09-11T05:23:28.433745+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:28.433745+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:36.350 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:36.738 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:36.738 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30419/6 objects degraded (506983.333%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":694,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1458245632,"bytes_avail":288452046848,"bytes_total":289910292480,"degraded_objects":30419,"degraded_total":6,"degraded_ratio":5069.833333333333,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":442,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":204},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:36.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:36.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:37.070 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:37.071 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:37.087 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-11T05:23:35.652995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12094,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1424068,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281691452,"statfs":{"total":289910292480,"available":288452046848,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1282853889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10051608,"num_objects":-2455,"num_object_clones":0,"num_object_copies":-4910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003645"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":23376,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:33.426073+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:33.426073+0000","last_peered":"2024-09-11T05:23:33.426073+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:33.426073+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.406031+0000","last_peered":"2024-09-11T05:23:28.406031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.406031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12093,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451428,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920412,"statfs":{"total":96636764160,"available":96174501888,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461762219},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215236,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:40.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:40.446 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-11T05:23:39.654329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12499,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446276,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669244,"statfs":{"total":289910292480,"available":288429305856,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8851480,"num_objects":-2162,"num_object_clones":0,"num_object_copies":-4324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003998"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":24186,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:38.423102+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:38.423102+0000","last_peered":"2024-09-11T05:23:38.423102+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:38.423102+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.406031+0000","last_peered":"2024-09-11T05:23:28.406031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.406031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12498,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:40.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:40.774 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:40.774 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:40.793 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-11T05:23:39.654329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12499,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446276,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669244,"statfs":{"total":289910292480,"available":288429305856,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8851480,"num_objects":-2162,"num_object_clones":0,"num_object_copies":-4324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003998"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":24186,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:38.423102+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:38.423102+0000","last_peered":"2024-09-11T05:23:38.423102+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:38.423102+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.406031+0000","last_peered":"2024-09-11T05:23:28.406031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.406031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12498,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20279, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435999+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:28.435999+0000', 'last_peered': '2024-09-11T05:23:28.435999+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:28.435999+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'20202", 'reported_seq': 24186, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:38.423102+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:38.423102+0000', 'last_peered': '2024-09-11T05:23:38.423102+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:38.423102+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1658, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17367, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1920, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20113, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.393703+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:28.393703+0000', 'last_peered': '2024-09-11T05:23:28.393703+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:28.393703+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19848, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.365158+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:28.365158+0000', 'last_peered': '2024-09-11T05:23:28.365158+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:28.365158+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:40.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 20161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.406031+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:28.406031+0000', 'last_peered': '2024-09-11T05:23:28.406031+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:28.406031+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3522}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20286, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435945+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:28.435945+0000', 'last_peered': '2024-09-11T05:23:28.435945+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:28.435945+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19798, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.331223+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:28.331223+0000', 'last_peered': '2024-09-11T05:23:28.331223+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:28.331223+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19946, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407386+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:28.407386+0000', 'last_peered': '2024-09-11T05:23:28.407386+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:28.407386+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:40.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19745, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.433745+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:28.433745+0000', 'last_peered': '2024-09-11T05:23:28.433745+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:28.433745+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:40.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:41.179 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:41.179 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30014/6 objects degraded (500233.333%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1480986624,"bytes_avail":288429305856,"bytes_total":289910292480,"degraded_objects":30014,"degraded_total":6,"degraded_ratio":5002.333333333333,"recovering_objects_per_sec":70,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":180},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:41.198 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:41.199 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:41.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:41.519 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:41.538 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-11T05:23:39.654329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12499,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446276,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669244,"statfs":{"total":289910292480,"available":288429305856,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8851480,"num_objects":-2162,"num_object_clones":0,"num_object_copies":-4324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003998"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":24186,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:38.423102+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:38.423102+0000","last_peered":"2024-09-11T05:23:38.423102+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:38.423102+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753225+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753225+0000","last_peered":"2024-09-11T05:22:17.753225+0000","last_clean":"2024-09-11T05:22:17.753225+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753225+0000","last_undegraded":"2024-09-11T05:22:17.753225+0000","last_fullsized":"2024-09-11T05:22:17.753225+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.406031+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.406031+0000","last_peered":"2024-09-11T05:23:28.406031+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.406031+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753594+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753594+0000","last_peered":"2024-09-11T05:22:17.753594+0000","last_clean":"2024-09-11T05:22:17.753594+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753594+0000","last_undegraded":"2024-09-11T05:22:17.753594+0000","last_fullsized":"2024-09-11T05:22:17.753594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12498,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:44.540 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:44.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:44.879 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:44.897 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-11T05:23:43.656007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12939,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450436,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281665084,"statfs":{"total":289910292480,"available":288425046016,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-639000,"num_objects":-157,"num_object_clones":0,"num_object_copies":-314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":157,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004427"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":25066,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:43.424885+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:43.424885+0000","last_peered":"2024-09-11T05:23:43.424885+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:43.424885+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2360,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12938,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477796,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894044,"statfs":{"total":96636764160,"available":96147501056,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:44.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:45.227 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:45.227 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:45.246 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-11T05:23:43.656007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12939,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450436,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281665084,"statfs":{"total":289910292480,"available":288425046016,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-639000,"num_objects":-157,"num_object_clones":0,"num_object_copies":-314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":157,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004427"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":25066,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:43.424885+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:43.424885+0000","last_peered":"2024-09-11T05:23:43.424885+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:43.424885+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2360,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12938,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477796,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894044,"statfs":{"total":96636764160,"available":96147501056,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:45.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20279, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435999+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:28.435999+0000', 'last_peered': '2024-09-11T05:23:28.435999+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:28.435999+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'20202", 'reported_seq': 25066, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:43.424885+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:43.424885+0000', 'last_peered': '2024-09-11T05:23:43.424885+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:43.424885+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1218, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17367, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2360, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1218}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20113, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.393703+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:28.393703+0000', 'last_peered': '2024-09-11T05:23:28.393703+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:28.393703+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19848, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.365158+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:28.365158+0000', 'last_peered': '2024-09-11T05:23:28.365158+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:28.365158+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 20162, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407322+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:28.407322+0000', 'last_peered': '2024-09-11T05:23:28.407322+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:28.407322+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3522}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:45.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:45.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20286, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435945+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:28.435945+0000', 'last_peered': '2024-09-11T05:23:28.435945+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:28.435945+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:45.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:45.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19798, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.331223+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:28.331223+0000', 'last_peered': '2024-09-11T05:23:28.331223+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:28.331223+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:45.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:45.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19946, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407386+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:28.407386+0000', 'last_peered': '2024-09-11T05:23:28.407386+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:28.407386+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:45.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:45.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19745, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.433745+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:28.433745+0000', 'last_peered': '2024-09-11T05:23:28.433745+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:28.433745+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:45.249 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:45.762 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:45.762 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29574/6 objects degraded (492900.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1485246464,"bytes_avail":288425046016,"bytes_total":289910292480,"degraded_objects":29574,"degraded_total":6,"degraded_ratio":4929,"recovering_objects_per_sec":101,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1214,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":13},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:45.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:45.785 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:46.086 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:46.102 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-11T05:23:45.763917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12939,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450436,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281665084,"statfs":{"total":289910292480,"available":288425046016,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-845,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.111908"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":25066,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:43.424885+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:43.424885+0000","last_peered":"2024-09-11T05:23:43.424885+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:43.424885+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2360,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12938,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477796,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894044,"statfs":{"total":96636764160,"available":96147501056,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:49.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:49.662 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:49.662 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:49.687 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-11T05:23:47.765382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12939,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433796,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681724,"statfs":{"total":289910292480,"available":288442085376,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-845,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.112387"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":25066,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:43.424885+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:43.424885+0000","last_peered":"2024-09-11T05:23:43.424885+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:43.424885+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2360,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.331223+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.331223+0000","last_peered":"2024-09-11T05:23:28.331223+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.331223+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29574,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12938,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:49.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:50.008 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:50.008 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:50.028 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-11T05:23:49.766053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13394,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433796,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681724,"statfs":{"total":289910292480,"available":288442085376,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.112463"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":25976,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:48.434129+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:48.434129+0000","last_peered":"2024-09-11T05:23:48.434129+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:48.434129+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2815,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19799,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.351154+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.351154+0000","last_peered":"2024-09-11T05:23:28.351154+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.351154+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13393,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215239,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:50.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:50.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20279, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435999+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:28.435999+0000', 'last_peered': '2024-09-11T05:23:28.435999+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:28.435999+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'20202", 'reported_seq': 25976, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:48.434129+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:48.434129+0000', 'last_peered': '2024-09-11T05:23:48.434129+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:48.434129+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 763, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17367, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2815, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -763}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20113, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.393703+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:28.393703+0000', 'last_peered': '2024-09-11T05:23:28.393703+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:28.393703+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19848, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.365158+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:28.365158+0000', 'last_peered': '2024-09-11T05:23:28.365158+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:28.365158+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 20162, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407322+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:28.407322+0000', 'last_peered': '2024-09-11T05:23:28.407322+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:28.407322+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3522}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:50.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20286, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435945+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:28.435945+0000', 'last_peered': '2024-09-11T05:23:28.435945+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:28.435945+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:50.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:50.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19799, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.351154+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:28.351154+0000', 'last_peered': '2024-09-11T05:23:28.351154+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:28.351154+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:50.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:50.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19946, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407386+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:28.407386+0000', 'last_peered': '2024-09-11T05:23:28.407386+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:28.407386+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:50.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:50.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19745, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.433745+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:28.433745+0000', 'last_peered': '2024-09-11T05:23:28.433745+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:28.433745+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:50.031 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:50.398 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:50.398 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29119/6 objects degraded (485316.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":708,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1468207104,"bytes_avail":288442085376,"bytes_total":289910292480,"degraded_objects":29119,"degraded_total":6,"degraded_ratio":4853.166666666667,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:50.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:50.417 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:50.750 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:50.751 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:50.768 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-11T05:23:49.766053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13394,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433796,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681724,"statfs":{"total":289910292480,"available":288442085376,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.112463"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.294145+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.294145+0000","last_peered":"2024-09-11T05:23:28.294145+0000","last_clean":"2024-09-11T05:23:28.294145+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.294145+0000","last_undegraded":"2024-09-11T05:23:28.294145+0000","last_fullsized":"2024-09-11T05:23:28.294145+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":25976,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:48.434129+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:48.434129+0000","last_peered":"2024-09-11T05:23:48.434129+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:48.434129+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2815,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.393703+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.393703+0000","last_peered":"2024-09-11T05:23:28.393703+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.393703+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26804,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.420257+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.420257+0000","last_peered":"2024-09-11T05:23:28.420257+0000","last_clean":"2024-09-11T05:23:28.420257+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.420257+0000","last_undegraded":"2024-09-11T05:23:28.420257+0000","last_fullsized":"2024-09-11T05:23:28.420257+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19799,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.351154+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.351154+0000","last_peered":"2024-09-11T05:23:28.351154+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.351154+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13393,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215239,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:53.770 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:54.090 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:54.091 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:54.109 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-11T05:23:53.767825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14007,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442116,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673404,"statfs":{"total":289910292480,"available":288433565696,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1508,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.112710"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.367014+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.367014+0000","last_peered":"2024-09-11T05:23:28.367014+0000","last_clean":"2024-09-11T05:23:28.367014+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.367014+0000","last_undegraded":"2024-09-11T05:23:28.367014+0000","last_fullsized":"2024-09-11T05:23:28.367014+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":27202,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:53.431832+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:53.431832+0000","last_peered":"2024-09-11T05:23:53.431832+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:53.431832+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3428,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.397757+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.397757+0000","last_peered":"2024-09-11T05:23:28.397757+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.397757+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26805,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.424977+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.424977+0000","last_peered":"2024-09-11T05:23:28.424977+0000","last_clean":"2024-09-11T05:23:28.424977+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.424977+0000","last_undegraded":"2024-09-11T05:23:28.424977+0000","last_fullsized":"2024-09-11T05:23:28.424977+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19799,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.351154+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.351154+0000","last_peered":"2024-09-11T05:23:28.351154+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.351154+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14006,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469476,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902364,"statfs":{"total":96636764160,"available":96156020736,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:54.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:54.416 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:54.416 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:54.435 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-11T05:23:53.767825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14007,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442116,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673404,"statfs":{"total":289910292480,"available":288433565696,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1508,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.112710"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.367014+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.367014+0000","last_peered":"2024-09-11T05:23:28.367014+0000","last_clean":"2024-09-11T05:23:28.367014+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.367014+0000","last_undegraded":"2024-09-11T05:23:28.367014+0000","last_fullsized":"2024-09-11T05:23:28.367014+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":27202,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:53.431832+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:53.431832+0000","last_peered":"2024-09-11T05:23:53.431832+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:53.431832+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3428,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.397757+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.397757+0000","last_peered":"2024-09-11T05:23:28.397757+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.397757+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26805,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.424977+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.424977+0000","last_peered":"2024-09-11T05:23:28.424977+0000","last_clean":"2024-09-11T05:23:28.424977+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.424977+0000","last_undegraded":"2024-09-11T05:23:28.424977+0000","last_fullsized":"2024-09-11T05:23:28.424977+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19799,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.351154+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.351154+0000","last_peered":"2024-09-11T05:23:28.351154+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.351154+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14006,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469476,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902364,"statfs":{"total":96636764160,"available":96156020736,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:54.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:54.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20279, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435999+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:28.435999+0000', 'last_peered': '2024-09-11T05:23:28.435999+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:28.435999+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:54.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T05:23:54.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'20202", 'reported_seq': 27202, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:53.431832+0000', 'last_change': '2024-09-11T05:22:15.824996+0000', 'last_active': '2024-09-11T05:23:53.431832+0000', 'last_peered': '2024-09-11T05:23:53.431832+0000', 'last_clean': '2024-09-11T05:14:12.136459+0000', 'last_became_active': '2024-09-11T05:14:14.241608+0000', 'last_became_peered': '2024-09-11T05:14:14.241608+0000', 'last_unstale': '2024-09-11T05:23:53.431832+0000', 'last_undegraded': '2024-09-11T05:14:13.863748+0000', 'last_fullsized': '2024-09-11T05:14:13.741556+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2868", 'last_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:00.287571+0000', 'objects_scrubbed': 2829, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1083, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17367, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3428, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -150}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:54.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:54.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.397757+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:28.397757+0000', 'last_peered': '2024-09-11T05:23:28.397757+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:28.397757+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:54.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:54.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19848, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.365158+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:28.365158+0000', 'last_peered': '2024-09-11T05:23:28.365158+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:28.365158+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:54.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:54.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 20162, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407322+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:28.407322+0000', 'last_peered': '2024-09-11T05:23:28.407322+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:28.407322+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3522}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:54.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:54.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20286, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435945+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:28.435945+0000', 'last_peered': '2024-09-11T05:23:28.435945+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:28.435945+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:54.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:54.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19799, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.351154+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:28.351154+0000', 'last_peered': '2024-09-11T05:23:28.351154+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:28.351154+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:54.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:54.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19946, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407386+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:28.407386+0000', 'last_peered': '2024-09-11T05:23:28.407386+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:28.407386+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:54.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:54.441 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19745, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.433745+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:28.433745+0000', 'last_peered': '2024-09-11T05:23:28.433745+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:28.433745+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:54.441 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:54.820 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28506/6 objects degraded (475100.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":712,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1476726784,"bytes_avail":288433565696,"bytes_total":289910292480,"degraded_objects":28506,"degraded_total":6,"degraded_ratio":4751,"recovering_objects_per_sec":124,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1508,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:54.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:54.838 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:55.256 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-11T05:23:54.880660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14007,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442116,"kb_used_data":171220,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673404,"statfs":{"total":289910292480,"available":288433565696,"internally_reserved":0,"allocated":175329280,"data_stored":174441559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1068,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.224653"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.427851+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:28.427851+0000","last_peered":"2024-09-11T05:23:28.427851+0000","last_clean":"2024-09-11T05:23:28.427851+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:28.427851+0000","last_undegraded":"2024-09-11T05:23:28.427851+0000","last_fullsized":"2024-09-11T05:23:28.427851+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.367014+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:28.367014+0000","last_peered":"2024-09-11T05:23:28.367014+0000","last_clean":"2024-09-11T05:23:28.367014+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:28.367014+0000","last_undegraded":"2024-09-11T05:23:28.367014+0000","last_fullsized":"2024-09-11T05:23:28.367014+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":27202,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:53.431832+0000","last_change":"2024-09-11T05:22:15.824996+0000","last_active":"2024-09-11T05:23:53.431832+0000","last_peered":"2024-09-11T05:23:53.431832+0000","last_clean":"2024-09-11T05:14:12.136459+0000","last_became_active":"2024-09-11T05:14:14.241608+0000","last_became_peered":"2024-09-11T05:14:14.241608+0000","last_unstale":"2024-09-11T05:23:53.431832+0000","last_undegraded":"2024-09-11T05:14:13.863748+0000","last_fullsized":"2024-09-11T05:14:13.741556+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3428,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750351+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:22:17.750351+0000","last_peered":"2024-09-11T05:22:17.750351+0000","last_clean":"2024-09-11T05:22:17.750351+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:22:17.750351+0000","last_undegraded":"2024-09-11T05:22:17.750351+0000","last_fullsized":"2024-09-11T05:22:17.750351+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.397757+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.397757+0000","last_peered":"2024-09-11T05:23:28.397757+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.397757+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750478+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:22:17.750478+0000","last_peered":"2024-09-11T05:22:17.750478+0000","last_clean":"2024-09-11T05:22:17.750478+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:22:17.750478+0000","last_undegraded":"2024-09-11T05:22:17.750478+0000","last_fullsized":"2024-09-11T05:22:17.750478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26805,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.424977+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.424977+0000","last_peered":"2024-09-11T05:23:28.424977+0000","last_clean":"2024-09-11T05:23:28.424977+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.424977+0000","last_undegraded":"2024-09-11T05:23:28.424977+0000","last_fullsized":"2024-09-11T05:23:28.424977+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750807+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:22:17.750807+0000","last_peered":"2024-09-11T05:22:17.750807+0000","last_clean":"2024-09-11T05:22:17.750807+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:22:17.750807+0000","last_undegraded":"2024-09-11T05:22:17.750807+0000","last_fullsized":"2024-09-11T05:22:17.750807+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19799,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.351154+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.351154+0000","last_peered":"2024-09-11T05:23:28.351154+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.351154+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750978+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:22:17.750978+0000","last_peered":"2024-09-11T05:22:17.750978+0000","last_clean":"2024-09-11T05:22:17.750978+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:22:17.750978+0000","last_undegraded":"2024-09-11T05:22:17.750978+0000","last_fullsized":"2024-09-11T05:22:17.750978+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.750575+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:22:17.750575+0000","last_peered":"2024-09-11T05:22:17.750575+0000","last_clean":"2024-09-11T05:22:17.750575+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:22:17.750575+0000","last_undegraded":"2024-09-11T05:22:17.750575+0000","last_fullsized":"2024-09-11T05:22:17.750575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28506,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14006,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174092288,"data_stored":174029666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469476,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902364,"statfs":{"total":96636764160,"available":96156020736,"internally_reserved":0,"allocated":479232,"data_stored":147176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:58.259 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:58.808 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:58.828 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-11T05:23:57.769245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14157,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446292,"kb_used_data":171228,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669228,"statfs":{"total":289910292480,"available":288429289472,"internally_reserved":0,"allocated":175337472,"data_stored":174446164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":763,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003191"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.397757+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.397757+0000","last_peered":"2024-09-11T05:23:28.397757+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.397757+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26805,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.424977+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.424977+0000","last_peered":"2024-09-11T05:23:28.424977+0000","last_clean":"2024-09-11T05:23:28.424977+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.424977+0000","last_undegraded":"2024-09-11T05:23:28.424977+0000","last_fullsized":"2024-09-11T05:23:28.424977+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19799,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.351154+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.351154+0000","last_peered":"2024-09-11T05:23:28.351154+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.351154+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14156,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473652,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898188,"statfs":{"total":96636764160,"available":96151744512,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:58.830 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:59.176 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:23:59.201 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-11T05:23:57.769245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14157,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446292,"kb_used_data":171228,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669228,"statfs":{"total":289910292480,"available":288429289472,"internally_reserved":0,"allocated":175337472,"data_stored":174446164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":763,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003191"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20279,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435999+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:28.435999+0000","last_peered":"2024-09-11T05:23:28.435999+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:28.435999+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.333797+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:28.333797+0000","last_peered":"2024-09-11T05:23:28.333797+0000","last_clean":"2024-09-11T05:23:28.333797+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:28.333797+0000","last_undegraded":"2024-09-11T05:23:28.333797+0000","last_fullsized":"2024-09-11T05:23:28.333797+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20042,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.412904+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:28.412904+0000","last_peered":"2024-09-11T05:23:28.412904+0000","last_clean":"2024-09-11T05:23:28.412904+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:28.412904+0000","last_undegraded":"2024-09-11T05:23:28.412904+0000","last_fullsized":"2024-09-11T05:23:28.412904+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20309,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.392734+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:28.392734+0000","last_peered":"2024-09-11T05:23:28.392734+0000","last_clean":"2024-09-11T05:23:28.392734+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:28.392734+0000","last_undegraded":"2024-09-11T05:23:28.392734+0000","last_fullsized":"2024-09-11T05:23:28.392734+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20019,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.364651+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:28.364651+0000","last_peered":"2024-09-11T05:23:28.364651+0000","last_clean":"2024-09-11T05:23:28.364651+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:28.364651+0000","last_undegraded":"2024-09-11T05:23:28.364651+0000","last_fullsized":"2024-09-11T05:23:28.364651+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.397757+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:28.397757+0000","last_peered":"2024-09-11T05:23:28.397757+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:28.397757+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753270+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:22:17.753270+0000","last_peered":"2024-09-11T05:22:17.753270+0000","last_clean":"2024-09-11T05:22:17.753270+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:22:17.753270+0000","last_undegraded":"2024-09-11T05:22:17.753270+0000","last_fullsized":"2024-09-11T05:22:17.753270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19848,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.365158+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:28.365158+0000","last_peered":"2024-09-11T05:23:28.365158+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:28.365158+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.760575+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:22:17.760575+0000","last_peered":"2024-09-11T05:22:17.760575+0000","last_clean":"2024-09-11T05:22:17.760575+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:22:17.760575+0000","last_undegraded":"2024-09-11T05:22:17.760575+0000","last_fullsized":"2024-09-11T05:22:17.760575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20162,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407322+0000","last_change":"2024-09-11T05:14:14.263434+0000","last_active":"2024-09-11T05:23:28.407322+0000","last_peered":"2024-09-11T05:23:28.407322+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:28.407322+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26805,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:23:28.424977+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:28.424977+0000","last_peered":"2024-09-11T05:23:28.424977+0000","last_clean":"2024-09-11T05:23:28.424977+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:28.424977+0000","last_undegraded":"2024-09-11T05:23:28.424977+0000","last_fullsized":"2024-09-11T05:23:28.424977+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20286,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.435945+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:28.435945+0000","last_peered":"2024-09-11T05:23:28.435945+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:28.435945+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19799,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.351154+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:28.351154+0000","last_peered":"2024-09-11T05:23:28.351154+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:28.351154+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T05:22:17.753632+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:22:17.753632+0000","last_peered":"2024-09-11T05:22:17.753632+0000","last_clean":"2024-09-11T05:22:17.753632+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:22:17.753632+0000","last_undegraded":"2024-09-11T05:22:17.753632+0000","last_fullsized":"2024-09-11T05:22:17.753632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19946,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.407386+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:28.407386+0000","last_peered":"2024-09-11T05:23:28.407386+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:28.407386+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19745,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:28.433745+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:28.433745+0000","last_peered":"2024-09-11T05:23:28.433745+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:28.433745+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14156,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461100,"kb_used_data":114268,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910740,"statfs":{"total":96636764160,"available":96164597760,"internally_reserved":0,"allocated":117010432,"data_stored":116734638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473652,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898188,"statfs":{"total":96636764160,"available":96151744512,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511540,"kb_used_data":56484,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860300,"statfs":{"total":96636764160,"available":96112947200,"internally_reserved":0,"allocated":57839616,"data_stored":57559745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20279, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435999+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:28.435999+0000', 'last_peered': '2024-09-11T05:23:28.435999+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:28.435999+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.397757+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:28.397757+0000', 'last_peered': '2024-09-11T05:23:28.397757+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:28.397757+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19848, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.365158+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:28.365158+0000', 'last_peered': '2024-09-11T05:23:28.365158+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:28.365158+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 20162, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407322+0000', 'last_change': '2024-09-11T05:14:14.263434+0000', 'last_active': '2024-09-11T05:23:28.407322+0000', 'last_peered': '2024-09-11T05:23:28.407322+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:23:28.407322+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3524, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3522}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:23:59.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:23:59.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20286, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.435945+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:28.435945+0000', 'last_peered': '2024-09-11T05:23:28.435945+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:28.435945+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:59.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:23:59.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19799, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.351154+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:28.351154+0000', 'last_peered': '2024-09-11T05:23:28.351154+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:28.351154+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:59.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:23:59.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19946, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.407386+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:28.407386+0000', 'last_peered': '2024-09-11T05:23:28.407386+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:28.407386+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:59.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:23:59.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19745, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:28.433745+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:28.433745+0000', 'last_peered': '2024-09-11T05:23:28.433745+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:28.433745+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:23:59.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:23:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:59.606 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28356/6 objects degraded (472600.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":717,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1481003008,"bytes_avail":288429289472,"bytes_total":289910292480,"degraded_objects":28356,"degraded_total":6,"degraded_ratio":4726,"recovering_objects_per_sec":95,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":763,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:23:59.626 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:23:59.627 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:23:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:23:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:00.001 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-11T05:23:59.769673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14570,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446336,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669184,"statfs":{"total":289910292480,"available":288429244416,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1176,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002473"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20993,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:57.993040+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:23:57.993040+0000","last_peered":"2024-09-11T05:23:57.993040+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:57.993040+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14569,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473652,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898188,"statfs":{"total":96636764160,"available":96151744512,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:03.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:03.650 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:03.671 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-11T05:24:01.770182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14570,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446336,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669184,"statfs":{"total":289910292480,"available":288429244416,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":563,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002357"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":20993,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:57.993040+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:23:57.993040+0000","last_peered":"2024-09-11T05:23:57.993040+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:23:57.993040+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14569,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473652,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898188,"statfs":{"total":96636764160,"available":96151744512,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:03.672 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:04.010 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:04.011 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:04.029 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-11T05:24:03.770842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15309,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433848,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681672,"statfs":{"total":289910292480,"available":288442032128,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.890181"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":22471,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:02.997972+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:02.997972+0000","last_peered":"2024-09-11T05:24:02.997972+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:02.997972+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15308,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:04.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:04.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20283, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896940+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:55.896940+0000', 'last_peered': '2024-09-11T05:23:55.896940+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:55.896940+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896412+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:55.896412+0000', 'last_peered': '2024-09-11T05:23:55.896412+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:55.896412+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19852, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894971+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:55.894971+0000', 'last_peered': '2024-09-11T05:23:55.894971+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:55.894971+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 22471, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:02.997972+0000', 'last_change': '2024-09-11T05:23:54.369072+0000', 'last_active': '2024-09-11T05:24:02.997972+0000', 'last_peered': '2024-09-11T05:24:02.997972+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:24:02.997972+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1152, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2372}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20291, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896065+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:55.896065+0000', 'last_peered': '2024-09-11T05:23:55.896065+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:55.896065+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:04.031 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19803, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.895081+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:55.895081+0000', 'last_peered': '2024-09-11T05:23:55.895081+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:55.895081+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:04.032 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:04.032 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19950, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897712+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:55.897712+0000', 'last_peered': '2024-09-11T05:23:55.897712+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:55.897712+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:04.032 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:04.032 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19749, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894770+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:55.894770+0000', 'last_peered': '2024-09-11T05:23:55.894770+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:55.894770+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:04.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:04.406 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:04.407 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27204/6 objects degraded (453400.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":722,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1468260352,"bytes_avail":288442032128,"bytes_total":289910292480,"degraded_objects":27204,"degraded_total":6,"degraded_ratio":4534,"recovering_objects_per_sec":146,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:04.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:04.429 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:04.726 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:04.727 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:04.742 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-11T05:24:03.770842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15309,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433848,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681672,"statfs":{"total":289910292480,"available":288442032128,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.890181"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":22471,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:02.997972+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:02.997972+0000","last_peered":"2024-09-11T05:24:02.997972+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:02.997972+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15308,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:07.745 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:08.073 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:08.073 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:08.090 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-11T05:24:07.772401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15309,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438008,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677512,"statfs":{"total":289910292480,"available":288437772288,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.885465"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":22471,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:02.997972+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:02.997972+0000","last_peered":"2024-09-11T05:24:02.997972+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:02.997972+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15308,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:08.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:08.437 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:08.456 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-11T05:24:07.772401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15309,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438008,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677512,"statfs":{"total":289910292480,"available":288437772288,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.885465"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":22471,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:02.997972+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:02.997972+0000","last_peered":"2024-09-11T05:24:02.997972+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:02.997972+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15308,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:08.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:08.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20283, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896940+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:55.896940+0000', 'last_peered': '2024-09-11T05:23:55.896940+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:55.896940+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:08.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:08.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896412+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:55.896412+0000', 'last_peered': '2024-09-11T05:23:55.896412+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:55.896412+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:08.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:08.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19852, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894971+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:55.894971+0000', 'last_peered': '2024-09-11T05:23:55.894971+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:55.894971+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 22471, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:02.997972+0000', 'last_change': '2024-09-11T05:23:54.369072+0000', 'last_active': '2024-09-11T05:24:02.997972+0000', 'last_peered': '2024-09-11T05:24:02.997972+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:24:02.997972+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2372, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1152, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2372}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20291, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896065+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:55.896065+0000', 'last_peered': '2024-09-11T05:23:55.896065+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:55.896065+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19803, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.895081+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:55.895081+0000', 'last_peered': '2024-09-11T05:23:55.895081+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:55.895081+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19950, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897712+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:55.897712+0000', 'last_peered': '2024-09-11T05:23:55.897712+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:55.897712+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:08.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19749, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894770+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:55.894770+0000', 'last_peered': '2024-09-11T05:23:55.894770+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:55.894770+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:08.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:08.835 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:08.836 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27204/6 objects degraded (453400.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":726,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1472520192,"bytes_avail":288437772288,"bytes_total":289910292480,"degraded_objects":27204,"degraded_total":6,"degraded_ratio":4534,"recovering_objects_per_sec":109,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:08.854 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:08.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:09.184 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:09.184 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:09.202 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-11T05:24:07.772401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15309,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438008,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677512,"statfs":{"total":289910292480,"available":288437772288,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.885465"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":22471,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:02.997972+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:02.997972+0000","last_peered":"2024-09-11T05:24:02.997972+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:02.997972+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15308,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:12.204 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:12.535 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:12.535 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:12.553 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-11T05:24:11.773886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15730,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438008,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677512,"statfs":{"total":289910292480,"available":288437772288,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004213"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":23313,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:07.997025+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:07.997025+0000","last_peered":"2024-09-11T05:24:07.997025+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:07.997025+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1951}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15729,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215243,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:12.555 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:12.878 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:12.878 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:12.897 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-11T05:24:11.773886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15730,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438008,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677512,"statfs":{"total":289910292480,"available":288437772288,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004213"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":23313,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:07.997025+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:07.997025+0000","last_peered":"2024-09-11T05:24:07.997025+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:07.997025+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1951}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15729,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215243,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:12.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:12.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20283, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896940+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:55.896940+0000', 'last_peered': '2024-09-11T05:23:55.896940+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:55.896940+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:12.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:12.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896412+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:55.896412+0000', 'last_peered': '2024-09-11T05:23:55.896412+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:55.896412+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:12.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:12.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19852, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894971+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:55.894971+0000', 'last_peered': '2024-09-11T05:23:55.894971+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:55.894971+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:12.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:24:12.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 23313, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:07.997025+0000', 'last_change': '2024-09-11T05:23:54.369072+0000', 'last_active': '2024-09-11T05:24:07.997025+0000', 'last_peered': '2024-09-11T05:24:07.997025+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:24:07.997025+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1573, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1951}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:12.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:12.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20291, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896065+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:55.896065+0000', 'last_peered': '2024-09-11T05:23:55.896065+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:55.896065+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:12.901 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:12.901 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19803, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.895081+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:55.895081+0000', 'last_peered': '2024-09-11T05:23:55.895081+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:55.895081+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:12.901 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:12.901 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19950, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897712+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:55.897712+0000', 'last_peered': '2024-09-11T05:23:55.897712+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:55.897712+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:12.901 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:12.901 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19749, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894770+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:55.894770+0000', 'last_peered': '2024-09-11T05:23:55.894770+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:55.894770+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:12.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:13.321 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:13.321 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26783/6 objects degraded (446383.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":731,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1472520192,"bytes_avail":288437772288,"bytes_total":289910292480,"degraded_objects":26783,"degraded_total":6,"degraded_ratio":4463.833333333333,"recovering_objects_per_sec":96,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1160,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:13.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:13.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:13.664 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:13.665 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:13.683 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-11T05:24:11.773886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15730,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438008,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677512,"statfs":{"total":289910292480,"available":288437772288,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004213"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20283,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896940+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.896940+0000","last_peered":"2024-09-11T05:23:55.896940+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.896940+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":23313,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:07.997025+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:07.997025+0000","last_peered":"2024-09-11T05:24:07.997025+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:07.997025+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1951}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15729,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215243,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:16.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:17.062 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:17.062 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:17.080 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-11T05:24:15.775536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26371,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16142,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442168,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673352,"statfs":{"total":289910292480,"available":288433512448,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004693"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":24137,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:12.994325+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:12.994325+0000","last_peered":"2024-09-11T05:24:12.994325+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:12.994325+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1985,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26371,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16141,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215244,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:17.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:17.415 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:17.416 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:17.433 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-11T05:24:15.775536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26371,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16142,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442168,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673352,"statfs":{"total":289910292480,"available":288433512448,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004693"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890086+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890086+0000","last_peered":"2024-09-11T05:23:55.890086+0000","last_clean":"2024-09-11T05:23:55.890086+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890086+0000","last_undegraded":"2024-09-11T05:23:55.890086+0000","last_fullsized":"2024-09-11T05:23:55.890086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":24137,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:12.994325+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:12.994325+0000","last_peered":"2024-09-11T05:24:12.994325+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:12.994325+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1985,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26371,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16141,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469484,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902356,"statfs":{"total":96636764160,"available":96156012544,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215244,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:17.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20284, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897973+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:55.897973+0000', 'last_peered': '2024-09-11T05:23:55.897973+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:55.897973+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896412+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:55.896412+0000', 'last_peered': '2024-09-11T05:23:55.896412+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:55.896412+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19852, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894971+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:55.894971+0000', 'last_peered': '2024-09-11T05:23:55.894971+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:55.894971+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 24137, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:12.994325+0000', 'last_change': '2024-09-11T05:23:54.369072+0000', 'last_active': '2024-09-11T05:24:12.994325+0000', 'last_peered': '2024-09-11T05:24:12.994325+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:24:12.994325+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1539, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1985, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1539}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20291, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896065+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:55.896065+0000', 'last_peered': '2024-09-11T05:23:55.896065+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:55.896065+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:17.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:17.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19803, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.895081+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:55.895081+0000', 'last_peered': '2024-09-11T05:23:55.895081+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:55.895081+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:17.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:17.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19950, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897712+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:55.897712+0000', 'last_peered': '2024-09-11T05:23:55.897712+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:55.897712+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:17.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:17.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19749, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894770+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:55.894770+0000', 'last_peered': '2024-09-11T05:23:55.894770+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:55.894770+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:17.436 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:17.836 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:17.837 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26371/6 objects degraded (439516.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":735,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1476780032,"bytes_avail":288433512448,"bytes_total":289910292480,"degraded_objects":26371,"degraded_total":6,"degraded_ratio":4395.166666666667,"recovering_objects_per_sec":69,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":833,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:17.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:17.856 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:18.159 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:18.159 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:18.177 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-11T05:24:17.776226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26371,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16142,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446328,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669192,"statfs":{"total":289910292480,"available":288429252608,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004663"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":24137,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:12.994325+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:12.994325+0000","last_peered":"2024-09-11T05:24:12.994325+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:12.994325+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1985,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26371,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16141,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473644,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898196,"statfs":{"total":96636764160,"available":96151752704,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215244,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:21.179 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:21.500 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:21.501 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:21.519 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-11T05:24:19.776889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16614,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446328,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669192,"statfs":{"total":289910292480,"available":288429252608,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004487"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":25081,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:18.000625+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:18.000625+0000","last_peered":"2024-09-11T05:24:18.000625+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:18.000625+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2457,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16613,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473644,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898196,"statfs":{"total":96636764160,"available":96151752704,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:21.521 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:21.841 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:21.841 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:21.859 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-11T05:24:21.777579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16614,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446328,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669192,"statfs":{"total":289910292480,"available":288429252608,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004524"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":25081,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:18.000625+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:18.000625+0000","last_peered":"2024-09-11T05:24:18.000625+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:18.000625+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2457,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16613,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473644,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898196,"statfs":{"total":96636764160,"available":96151752704,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20284, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897973+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:55.897973+0000', 'last_peered': '2024-09-11T05:23:55.897973+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:55.897973+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:21.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896412+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:55.896412+0000', 'last_peered': '2024-09-11T05:23:55.896412+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:55.896412+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19852, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894971+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:55.894971+0000', 'last_peered': '2024-09-11T05:23:55.894971+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:55.894971+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 25081, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:18.000625+0000', 'last_change': '2024-09-11T05:23:54.369072+0000', 'last_active': '2024-09-11T05:24:18.000625+0000', 'last_peered': '2024-09-11T05:24:18.000625+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:24:18.000625+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2457, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1067}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20291, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896065+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:55.896065+0000', 'last_peered': '2024-09-11T05:23:55.896065+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:55.896065+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:21.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19803, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.895081+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:55.895081+0000', 'last_peered': '2024-09-11T05:23:55.895081+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:55.895081+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:21.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:21.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19950, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897712+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:55.897712+0000', 'last_peered': '2024-09-11T05:23:55.897712+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:55.897712+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:21.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:21.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19749, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894770+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:55.894770+0000', 'last_peered': '2024-09-11T05:23:55.894770+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:55.894770+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:21.862 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:22.268 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:22.269 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25899/6 objects degraded (431650.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":740,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1481039872,"bytes_avail":288429252608,"bytes_total":289910292480,"degraded_objects":25899,"degraded_total":6,"degraded_ratio":4316.5,"recovering_objects_per_sec":73,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":884,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:22.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:22.289 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:22.614 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:22.633 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-11T05:24:21.777579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16614,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446328,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669192,"statfs":{"total":289910292480,"available":288429252608,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004524"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":25081,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:18.000625+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:18.000625+0000","last_peered":"2024-09-11T05:24:18.000625+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:18.000625+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2457,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16613,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473644,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898196,"statfs":{"total":96636764160,"available":96151752704,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:25.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:25.995 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:25.995 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:26.013 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-11T05:24:25.778928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17172,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433848,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681672,"statfs":{"total":289910292480,"available":288442032128,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004412"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":26197,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:23.006220+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:23.006220+0000","last_peered":"2024-09-11T05:24:23.006220+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:23.006220+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":509,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3015,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17171,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215246,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:26.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:26.329 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:26.333 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:26.348 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-11T05:24:25.778928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17172,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433848,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681672,"statfs":{"total":289910292480,"available":288442032128,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004412"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":26197,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:23.006220+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:23.006220+0000","last_peered":"2024-09-11T05:24:23.006220+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:23.006220+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":509,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3015,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17171,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215246,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:26.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:26.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20284, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897973+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:23:55.897973+0000', 'last_peered': '2024-09-11T05:23:55.897973+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:23:55.897973+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:26.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:26.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896412+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:23:55.896412+0000', 'last_peered': '2024-09-11T05:23:55.896412+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:23:55.896412+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:26.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:26.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19852, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894971+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:23:55.894971+0000', 'last_peered': '2024-09-11T05:23:55.894971+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:23:55.894971+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 26197, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:23.006220+0000', 'last_change': '2024-09-11T05:23:54.369072+0000', 'last_active': '2024-09-11T05:24:23.006220+0000', 'last_peered': '2024-09-11T05:24:23.006220+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:24:23.006220+0000', 'last_undegraded': '2024-09-11T05:14:13.813095+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 509, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3015, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -509}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20291, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.896065+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:23:55.896065+0000', 'last_peered': '2024-09-11T05:23:55.896065+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:23:55.896065+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19803, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.895081+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:23:55.895081+0000', 'last_peered': '2024-09-11T05:23:55.895081+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:23:55.895081+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19950, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.897712+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:23:55.897712+0000', 'last_peered': '2024-09-11T05:23:55.897712+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:23:55.897712+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:26.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 19749, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:23:55.894770+0000', 'last_change': '2024-09-11T05:14:14.291923+0000', 'last_active': '2024-09-11T05:23:55.894770+0000', 'last_peered': '2024-09-11T05:23:55.894770+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:23:55.894770+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3518, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:26.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:26.749 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:26.749 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25341/6 objects degraded (422350.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":744,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1468260352,"bytes_avail":288442032128,"bytes_total":289910292480,"degraded_objects":25341,"degraded_total":6,"degraded_ratio":4223.5,"recovering_objects_per_sec":85,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1030,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:26.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:26.765 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:27.079 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:27.080 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:27.098 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-11T05:24:25.778928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17172,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433848,"kb_used_data":171260,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681672,"statfs":{"total":289910292480,"available":288442032128,"internally_reserved":0,"allocated":175370240,"data_stored":174457842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1030,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004412"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20284,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897973+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:23:55.897973+0000","last_peered":"2024-09-11T05:23:55.897973+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:23:55.897973+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20017,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.897175+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:23:55.897175+0000","last_peered":"2024-09-11T05:23:55.897175+0000","last_clean":"2024-09-11T05:23:55.897175+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:23:55.897175+0000","last_undegraded":"2024-09-11T05:23:55.897175+0000","last_fullsized":"2024-09-11T05:23:55.897175+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20046,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896678+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:23:55.896678+0000","last_peered":"2024-09-11T05:23:55.896678+0000","last_clean":"2024-09-11T05:23:55.896678+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:23:55.896678+0000","last_undegraded":"2024-09-11T05:23:55.896678+0000","last_fullsized":"2024-09-11T05:23:55.896678+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20314,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.900399+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:23:55.900399+0000","last_peered":"2024-09-11T05:23:55.900399+0000","last_clean":"2024-09-11T05:23:55.900399+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:23:55.900399+0000","last_undegraded":"2024-09-11T05:23:55.900399+0000","last_fullsized":"2024-09-11T05:23:55.900399+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20023,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894363+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:23:55.894363+0000","last_peered":"2024-09-11T05:23:55.894363+0000","last_clean":"2024-09-11T05:23:55.894363+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:23:55.894363+0000","last_undegraded":"2024-09-11T05:23:55.894363+0000","last_fullsized":"2024-09-11T05:23:55.894363+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896412+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:23:55.896412+0000","last_peered":"2024-09-11T05:23:55.896412+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:23:55.896412+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896471+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:23:55.896471+0000","last_peered":"2024-09-11T05:23:55.896471+0000","last_clean":"2024-09-11T05:23:55.896471+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:23:55.896471+0000","last_undegraded":"2024-09-11T05:23:55.896471+0000","last_fullsized":"2024-09-11T05:23:55.896471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19852,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894971+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:23:55.894971+0000","last_peered":"2024-09-11T05:23:55.894971+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:23:55.894971+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.894742+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:23:55.894742+0000","last_peered":"2024-09-11T05:23:55.894742+0000","last_clean":"2024-09-11T05:23:55.894742+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:23:55.894742+0000","last_undegraded":"2024-09-11T05:23:55.894742+0000","last_fullsized":"2024-09-11T05:23:55.894742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":26197,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:23.006220+0000","last_change":"2024-09-11T05:23:54.369072+0000","last_active":"2024-09-11T05:24:23.006220+0000","last_peered":"2024-09-11T05:24:23.006220+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:23.006220+0000","last_undegraded":"2024-09-11T05:14:13.813095+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":509,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3015,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-509}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.895658+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:23:55.895658+0000","last_peered":"2024-09-11T05:23:55.895658+0000","last_clean":"2024-09-11T05:23:55.895658+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:23:55.895658+0000","last_undegraded":"2024-09-11T05:23:55.895658+0000","last_fullsized":"2024-09-11T05:23:55.895658+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20291,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.896065+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:23:55.896065+0000","last_peered":"2024-09-11T05:23:55.896065+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:23:55.896065+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19803,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.895081+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:23:55.895081+0000","last_peered":"2024-09-11T05:23:55.895081+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:23:55.895081+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.896931+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:23:55.896931+0000","last_peered":"2024-09-11T05:23:55.896931+0000","last_clean":"2024-09-11T05:23:55.896931+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:23:55.896931+0000","last_undegraded":"2024-09-11T05:23:55.896931+0000","last_fullsized":"2024-09-11T05:23:55.896931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.897712+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:23:55.897712+0000","last_peered":"2024-09-11T05:23:55.897712+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:23:55.897712+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:23:55.894770+0000","last_change":"2024-09-11T05:14:14.291923+0000","last_active":"2024-09-11T05:23:55.894770+0000","last_peered":"2024-09-11T05:23:55.894770+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:23:55.894770+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3518,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17171,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174084096,"data_stored":174028432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461120,"kb_used_data":114284,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910720,"statfs":{"total":96636764160,"available":96164577280,"internally_reserved":0,"allocated":117026816,"data_stored":116740477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461164,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910676,"statfs":{"total":96636764160,"available":96164532224,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215246,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":56500,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":57856000,"data_stored":57565584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:30.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:30.431 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:30.432 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:30.450 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-11T05:24:29.780017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17835,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438020,"kb_used_data":171276,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677500,"statfs":{"total":289910292480,"available":288437760000,"internally_reserved":0,"allocated":175386624,"data_stored":174463652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002438"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20019,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.284681+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:27.284681+0000","last_peered":"2024-09-11T05:24:27.284681+0000","last_clean":"2024-09-11T05:24:27.284681+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:27.284681+0000","last_undegraded":"2024-09-11T05:24:27.284681+0000","last_fullsized":"2024-09-11T05:24:27.284681+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20048,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.283461+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:27.283461+0000","last_peered":"2024-09-11T05:24:27.283461+0000","last_clean":"2024-09-11T05:24:27.283461+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:27.283461+0000","last_undegraded":"2024-09-11T05:24:27.283461+0000","last_fullsized":"2024-09-11T05:24:27.283461+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20120,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:27.281543+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:27.281543+0000","last_peered":"2024-09-11T05:24:27.281543+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:27.281543+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.280704+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:27.280704+0000","last_peered":"2024-09-11T05:24:27.280704+0000","last_clean":"2024-09-11T05:24:27.280704+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:27.280704+0000","last_undegraded":"2024-09-11T05:24:27.280704+0000","last_fullsized":"2024-09-11T05:24:27.280704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19856,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286593+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.286593+0000","last_peered":"2024-09-11T05:24:28.286593+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.286593+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27217,"reported_epoch":51,"state":"peering","last_fresh":"2024-09-11T05:24:26.989207+0000","last_change":"2024-09-11T05:24:27.291831+0000","last_active":"2024-09-11T05:24:27.291831+0000","last_peered":"2024-09-11T05:24:26.989207+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:26.989207+0000","last_undegraded":"2024-09-11T05:24:26.989207+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26811,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.280349+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:27.280349+0000","last_peered":"2024-09-11T05:24:27.280349+0000","last_clean":"2024-09-11T05:24:27.280349+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:27.280349+0000","last_undegraded":"2024-09-11T05:24:27.280349+0000","last_fullsized":"2024-09-11T05:24:27.280349+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.285346+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:27.285346+0000","last_peered":"2024-09-11T05:24:27.285346+0000","last_clean":"2024-09-11T05:24:27.285346+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:27.285346+0000","last_undegraded":"2024-09-11T05:24:27.285346+0000","last_fullsized":"2024-09-11T05:24:27.285346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20062,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.435071+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:28.435071+0000","last_peered":"2024-09-11T05:24:28.435071+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:28.435071+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17834,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461128,"kb_used_data":114292,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910712,"statfs":{"total":96636764160,"available":96164569088,"internally_reserved":0,"allocated":117035008,"data_stored":116743382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511568,"kb_used_data":56508,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860272,"statfs":{"total":96636764160,"available":96112918528,"internally_reserved":0,"allocated":57864192,"data_stored":57568489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:30.452 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:30.767 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:30.769 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:30.787 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-11T05:24:29.780017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17835,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438020,"kb_used_data":171276,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677500,"statfs":{"total":289910292480,"available":288437760000,"internally_reserved":0,"allocated":175386624,"data_stored":174463652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002438"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20019,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.284681+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:27.284681+0000","last_peered":"2024-09-11T05:24:27.284681+0000","last_clean":"2024-09-11T05:24:27.284681+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:27.284681+0000","last_undegraded":"2024-09-11T05:24:27.284681+0000","last_fullsized":"2024-09-11T05:24:27.284681+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20048,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.283461+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:27.283461+0000","last_peered":"2024-09-11T05:24:27.283461+0000","last_clean":"2024-09-11T05:24:27.283461+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:27.283461+0000","last_undegraded":"2024-09-11T05:24:27.283461+0000","last_fullsized":"2024-09-11T05:24:27.283461+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20120,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:27.281543+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:27.281543+0000","last_peered":"2024-09-11T05:24:27.281543+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:27.281543+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.280704+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:27.280704+0000","last_peered":"2024-09-11T05:24:27.280704+0000","last_clean":"2024-09-11T05:24:27.280704+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:27.280704+0000","last_undegraded":"2024-09-11T05:24:27.280704+0000","last_fullsized":"2024-09-11T05:24:27.280704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19856,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286593+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.286593+0000","last_peered":"2024-09-11T05:24:28.286593+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.286593+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27217,"reported_epoch":51,"state":"peering","last_fresh":"2024-09-11T05:24:26.989207+0000","last_change":"2024-09-11T05:24:27.291831+0000","last_active":"2024-09-11T05:24:27.291831+0000","last_peered":"2024-09-11T05:24:26.989207+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:26.989207+0000","last_undegraded":"2024-09-11T05:24:26.989207+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26811,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.280349+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:27.280349+0000","last_peered":"2024-09-11T05:24:27.280349+0000","last_clean":"2024-09-11T05:24:27.280349+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:27.280349+0000","last_undegraded":"2024-09-11T05:24:27.280349+0000","last_fullsized":"2024-09-11T05:24:27.280349+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.285346+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:27.285346+0000","last_peered":"2024-09-11T05:24:27.285346+0000","last_clean":"2024-09-11T05:24:27.285346+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:27.285346+0000","last_undegraded":"2024-09-11T05:24:27.285346+0000","last_fullsized":"2024-09-11T05:24:27.285346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20062,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.435071+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:28.435071+0000","last_peered":"2024-09-11T05:24:28.435071+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:28.435071+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17834,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461128,"kb_used_data":114292,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910712,"statfs":{"total":96636764160,"available":96164569088,"internally_reserved":0,"allocated":117035008,"data_stored":116743382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511568,"kb_used_data":56508,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860272,"statfs":{"total":96636764160,"available":96112918528,"internally_reserved":0,"allocated":57864192,"data_stored":57568489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:30.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:30.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:30.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:30.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20120, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:27.281543+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:27.281543+0000', 'last_peered': '2024-09-11T05:24:27.281543+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:27.281543+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19856, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.286593+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.286593+0000', 'last_peered': '2024-09-11T05:24:28.286593+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.286593+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20033", 'reported_seq': 27217, 'reported_epoch': 51, 'state': 'peering', 'last_fresh': '2024-09-11T05:24:26.989207+0000', 'last_change': '2024-09-11T05:24:27.291831+0000', 'last_active': '2024-09-11T05:24:27.291831+0000', 'last_peered': '2024-09-11T05:24:26.989207+0000', 'last_clean': '2024-09-11T05:14:12.132002+0000', 'last_became_active': '2024-09-11T05:14:14.255755+0000', 'last_became_peered': '2024-09-11T05:14:14.255755+0000', 'last_unstale': '2024-09-11T05:24:26.989207+0000', 'last_undegraded': '2024-09-11T05:24:26.989207+0000', 'last_fullsized': '2024-09-11T05:14:13.741492+0000', 'mapping_epoch': 52, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1993", 'last_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.998266+0000', 'objects_scrubbed': 1966, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 723, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18062, 'num_write_kb': 32192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3524, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:30.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19954, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.286631+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.286631+0000', 'last_peered': '2024-09-11T05:24:28.286631+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.286631+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:30.790 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:30.790 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 20062, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.435071+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:28.435071+0000', 'last_peered': '2024-09-11T05:24:28.435071+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:28.435071+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3364, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 155, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:30.790 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:31.229 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:31.229 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24678/6 objects degraded (411300.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":749,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1472532480,"bytes_avail":288437760000,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":24678,"degraded_total":6,"degraded_ratio":4113,"recovering_objects_per_sec":152,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1221,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:31.251 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:31.251 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:31.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:31.569 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:31.586 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-11T05:24:29.780017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17835,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438020,"kb_used_data":171276,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677500,"statfs":{"total":289910292480,"available":288437760000,"internally_reserved":0,"allocated":175386624,"data_stored":174463652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002438"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20019,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.284681+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:27.284681+0000","last_peered":"2024-09-11T05:24:27.284681+0000","last_clean":"2024-09-11T05:24:27.284681+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:27.284681+0000","last_undegraded":"2024-09-11T05:24:27.284681+0000","last_fullsized":"2024-09-11T05:24:27.284681+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26859,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890781+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:23:55.890781+0000","last_peered":"2024-09-11T05:23:55.890781+0000","last_clean":"2024-09-11T05:23:55.890781+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:23:55.890781+0000","last_undegraded":"2024-09-11T05:23:55.890781+0000","last_fullsized":"2024-09-11T05:23:55.890781+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20048,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.283461+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:27.283461+0000","last_peered":"2024-09-11T05:24:27.283461+0000","last_clean":"2024-09-11T05:24:27.283461+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:27.283461+0000","last_undegraded":"2024-09-11T05:24:27.283461+0000","last_fullsized":"2024-09-11T05:24:27.283461+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27396,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890137+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:23:55.890137+0000","last_peered":"2024-09-11T05:23:55.890137+0000","last_clean":"2024-09-11T05:23:55.890137+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:23:55.890137+0000","last_undegraded":"2024-09-11T05:23:55.890137+0000","last_fullsized":"2024-09-11T05:23:55.890137+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.997238+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:23:55.997238+0000","last_peered":"2024-09-11T05:23:55.997238+0000","last_clean":"2024-09-11T05:23:55.997238+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:23:55.997238+0000","last_undegraded":"2024-09-11T05:23:55.997238+0000","last_fullsized":"2024-09-11T05:23:55.997238+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890207+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:23:55.890207+0000","last_peered":"2024-09-11T05:23:55.890207+0000","last_clean":"2024-09-11T05:23:55.890207+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:23:55.890207+0000","last_undegraded":"2024-09-11T05:23:55.890207+0000","last_fullsized":"2024-09-11T05:23:55.890207+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20120,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:27.281543+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:27.281543+0000","last_peered":"2024-09-11T05:24:27.281543+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:27.281543+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.280704+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:27.280704+0000","last_peered":"2024-09-11T05:24:27.280704+0000","last_clean":"2024-09-11T05:24:27.280704+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:27.280704+0000","last_undegraded":"2024-09-11T05:24:27.280704+0000","last_fullsized":"2024-09-11T05:24:27.280704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19856,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286593+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.286593+0000","last_peered":"2024-09-11T05:24:28.286593+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.286593+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27217,"reported_epoch":51,"state":"peering","last_fresh":"2024-09-11T05:24:26.989207+0000","last_change":"2024-09-11T05:24:27.291831+0000","last_active":"2024-09-11T05:24:27.291831+0000","last_peered":"2024-09-11T05:24:26.989207+0000","last_clean":"2024-09-11T05:14:12.132002+0000","last_became_active":"2024-09-11T05:14:14.255755+0000","last_became_peered":"2024-09-11T05:14:14.255755+0000","last_unstale":"2024-09-11T05:24:26.989207+0000","last_undegraded":"2024-09-11T05:24:26.989207+0000","last_fullsized":"2024-09-11T05:14:13.741492+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890324+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:23:55.890324+0000","last_peered":"2024-09-11T05:23:55.890324+0000","last_clean":"2024-09-11T05:23:55.890324+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:23:55.890324+0000","last_undegraded":"2024-09-11T05:23:55.890324+0000","last_fullsized":"2024-09-11T05:23:55.890324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26811,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.280349+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:27.280349+0000","last_peered":"2024-09-11T05:24:27.280349+0000","last_clean":"2024-09-11T05:24:27.280349+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:27.280349+0000","last_undegraded":"2024-09-11T05:24:27.280349+0000","last_fullsized":"2024-09-11T05:24:27.280349+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.890590+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:23:55.890590+0000","last_peered":"2024-09-11T05:23:55.890590+0000","last_clean":"2024-09-11T05:23:55.890590+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:23:55.890590+0000","last_undegraded":"2024-09-11T05:23:55.890590+0000","last_fullsized":"2024-09-11T05:23:55.890590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T05:24:27.285346+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:27.285346+0000","last_peered":"2024-09-11T05:24:27.285346+0000","last_clean":"2024-09-11T05:24:27.285346+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:27.285346+0000","last_undegraded":"2024-09-11T05:24:27.285346+0000","last_fullsized":"2024-09-11T05:24:27.285346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891101+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:23:55.891101+0000","last_peered":"2024-09-11T05:23:55.891101+0000","last_clean":"2024-09-11T05:23:55.891101+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:23:55.891101+0000","last_undegraded":"2024-09-11T05:23:55.891101+0000","last_fullsized":"2024-09-11T05:23:55.891101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T05:23:55.891366+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:23:55.891366+0000","last_peered":"2024-09-11T05:23:55.891366+0000","last_clean":"2024-09-11T05:23:55.891366+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:23:55.891366+0000","last_undegraded":"2024-09-11T05:23:55.891366+0000","last_fullsized":"2024-09-11T05:23:55.891366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20062,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.435071+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:28.435071+0000","last_peered":"2024-09-11T05:24:28.435071+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:28.435071+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17834,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461128,"kb_used_data":114292,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910712,"statfs":{"total":96636764160,"available":96164569088,"internally_reserved":0,"allocated":117035008,"data_stored":116743382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465324,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906516,"statfs":{"total":96636764160,"available":96160272384,"internally_reserved":0,"allocated":487424,"data_stored":151781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511568,"kb_used_data":56508,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860272,"statfs":{"total":96636764160,"available":96112918528,"internally_reserved":0,"allocated":57864192,"data_stored":57568489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:34.589 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:34.912 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:34.912 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:34.931 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-11T05:24:33.781297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18146,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442208,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673312,"statfs":{"total":289910292480,"available":288433471488,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002368"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26863,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295790+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295790+0000","last_peered":"2024-09-11T05:24:28.295790+0000","last_clean":"2024-09-11T05:24:28.295790+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295790+0000","last_undegraded":"2024-09-11T05:24:28.295790+0000","last_fullsized":"2024-09-11T05:24:28.295790+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295695+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295695+0000","last_peered":"2024-09-11T05:24:28.295695+0000","last_clean":"2024-09-11T05:24:28.295695+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295695+0000","last_undegraded":"2024-09-11T05:24:28.295695+0000","last_fullsized":"2024-09-11T05:24:28.295695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295786+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295786+0000","last_peered":"2024-09-11T05:24:28.295786+0000","last_clean":"2024-09-11T05:24:28.295786+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295786+0000","last_undegraded":"2024-09-11T05:24:28.295786+0000","last_fullsized":"2024-09-11T05:24:28.295786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20684,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:33.440223+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:33.440223+0000","last_peered":"2024-09-11T05:24:33.440223+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:33.440223+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18145,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:34.933 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:35.239 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:35.239 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:35.260 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-11T05:24:33.781297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18146,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442208,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673312,"statfs":{"total":289910292480,"available":288433471488,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002368"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26863,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295790+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295790+0000","last_peered":"2024-09-11T05:24:28.295790+0000","last_clean":"2024-09-11T05:24:28.295790+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295790+0000","last_undegraded":"2024-09-11T05:24:28.295790+0000","last_fullsized":"2024-09-11T05:24:28.295790+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295695+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295695+0000","last_peered":"2024-09-11T05:24:28.295695+0000","last_clean":"2024-09-11T05:24:28.295695+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295695+0000","last_undegraded":"2024-09-11T05:24:28.295695+0000","last_fullsized":"2024-09-11T05:24:28.295695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295786+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295786+0000","last_peered":"2024-09-11T05:24:28.295786+0000","last_clean":"2024-09-11T05:24:28.295786+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295786+0000","last_undegraded":"2024-09-11T05:24:28.295786+0000","last_fullsized":"2024-09-11T05:24:28.295786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20684,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:33.440223+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:33.440223+0000","last_peered":"2024-09-11T05:24:33.440223+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:33.440223+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18145,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:35.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:35.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:35.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:35.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:35.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:35.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19954, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.286631+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.286631+0000', 'last_peered': '2024-09-11T05:24:28.286631+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.286631+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:35.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 20684, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:33.440223+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:33.440223+0000', 'last_peered': '2024-09-11T05:24:33.440223+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:33.440223+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3053, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 466, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:35.262 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:35.653 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:35.653 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24367/6 objects degraded (406116.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":753,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1476820992,"bytes_avail":288433471488,"bytes_total":289910292480,"degraded_objects":24367,"degraded_total":6,"degraded_ratio":4061.1666666666661,"recovering_objects_per_sec":121,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:35.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:35.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:35.978 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:35.978 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:35.997 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-11T05:24:35.782182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18146,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442208,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673312,"statfs":{"total":289910292480,"available":288433471488,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.504995"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26863,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295790+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295790+0000","last_peered":"2024-09-11T05:24:28.295790+0000","last_clean":"2024-09-11T05:24:28.295790+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295790+0000","last_undegraded":"2024-09-11T05:24:28.295790+0000","last_fullsized":"2024-09-11T05:24:28.295790+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295695+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295695+0000","last_peered":"2024-09-11T05:24:28.295695+0000","last_clean":"2024-09-11T05:24:28.295695+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295695+0000","last_undegraded":"2024-09-11T05:24:28.295695+0000","last_fullsized":"2024-09-11T05:24:28.295695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295786+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295786+0000","last_peered":"2024-09-11T05:24:28.295786+0000","last_clean":"2024-09-11T05:24:28.295786+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295786+0000","last_undegraded":"2024-09-11T05:24:28.295786+0000","last_fullsized":"2024-09-11T05:24:28.295786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20684,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:33.440223+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:33.440223+0000","last_peered":"2024-09-11T05:24:33.440223+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:33.440223+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18145,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:38.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:39.328 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:39.328 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:39.345 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-11T05:24:37.782608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18146,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003263"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295695+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295695+0000","last_peered":"2024-09-11T05:24:28.295695+0000","last_clean":"2024-09-11T05:24:28.295695+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295695+0000","last_undegraded":"2024-09-11T05:24:28.295695+0000","last_fullsized":"2024-09-11T05:24:28.295695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295786+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295786+0000","last_peered":"2024-09-11T05:24:28.295786+0000","last_clean":"2024-09-11T05:24:28.295786+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295786+0000","last_undegraded":"2024-09-11T05:24:28.295786+0000","last_fullsized":"2024-09-11T05:24:28.295786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20684,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:33.440223+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:33.440223+0000","last_peered":"2024-09-11T05:24:33.440223+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:33.440223+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18145,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:39.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:39.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:39.652 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:39.670 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-11T05:24:37.782608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18146,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-974,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003263"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295695+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295695+0000","last_peered":"2024-09-11T05:24:28.295695+0000","last_clean":"2024-09-11T05:24:28.295695+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295695+0000","last_undegraded":"2024-09-11T05:24:28.295695+0000","last_fullsized":"2024-09-11T05:24:28.295695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295786+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295786+0000","last_peered":"2024-09-11T05:24:28.295786+0000","last_clean":"2024-09-11T05:24:28.295786+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295786+0000","last_undegraded":"2024-09-11T05:24:28.295786+0000","last_fullsized":"2024-09-11T05:24:28.295786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19954,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.286631+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.286631+0000","last_peered":"2024-09-11T05:24:28.286631+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.286631+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":20684,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:33.440223+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:33.440223+0000","last_peered":"2024-09-11T05:24:33.440223+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:33.440223+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18145,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:39.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:39.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:39.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19954, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.286631+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.286631+0000', 'last_peered': '2024-09-11T05:24:28.286631+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.286631+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:39.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:39.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 20684, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:33.440223+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:33.440223+0000', 'last_peered': '2024-09-11T05:24:33.440223+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:33.440223+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3053, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 466, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:39.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:40.049 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:40.049 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24036/6 objects degraded (400600.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":757,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1481080832,"bytes_avail":288429211648,"bytes_total":289910292480,"degraded_objects":24036,"degraded_total":6,"degraded_ratio":4006,"recovering_objects_per_sec":69,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":796,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:40.067 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:40.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:40.382 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:40.382 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:40.402 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-11T05:24:39.783223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24036,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18477,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.499587"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295695+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295695+0000","last_peered":"2024-09-11T05:24:28.295695+0000","last_clean":"2024-09-11T05:24:28.295695+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295695+0000","last_undegraded":"2024-09-11T05:24:28.295695+0000","last_fullsized":"2024-09-11T05:24:28.295695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295786+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295786+0000","last_peered":"2024-09-11T05:24:28.295786+0000","last_clean":"2024-09-11T05:24:28.295786+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295786+0000","last_undegraded":"2024-09-11T05:24:28.295786+0000","last_fullsized":"2024-09-11T05:24:28.295786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":21346,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:38.442292+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:38.442292+0000","last_peered":"2024-09-11T05:24:38.442292+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:38.442292+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2722,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":797,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2721}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24036,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18476,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215249,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:43.405 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:43.724 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:43.725 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:43.743 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-09-11T05:24:41.783707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24036,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18477,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003690"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295695+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295695+0000","last_peered":"2024-09-11T05:24:28.295695+0000","last_clean":"2024-09-11T05:24:28.295695+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295695+0000","last_undegraded":"2024-09-11T05:24:28.295695+0000","last_fullsized":"2024-09-11T05:24:28.295695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295786+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295786+0000","last_peered":"2024-09-11T05:24:28.295786+0000","last_clean":"2024-09-11T05:24:28.295786+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295786+0000","last_undegraded":"2024-09-11T05:24:28.295786+0000","last_fullsized":"2024-09-11T05:24:28.295786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":21346,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:38.442292+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:38.442292+0000","last_peered":"2024-09-11T05:24:38.442292+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:38.442292+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2722,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":797,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2721}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24036,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18476,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215249,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:43.745 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:44.081 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:44.101 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-11T05:24:43.784340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18813,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003672"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":22018,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:43.437823+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:43.437823+0000","last_peered":"2024-09-11T05:24:43.437823+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:43.437823+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18812,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:44.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:44.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:44.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:44.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:44.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:44.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:44.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:44.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:44.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:44.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:44.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:44.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19955, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.287800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.287800+0000', 'last_peered': '2024-09-11T05:24:28.287800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.287800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:44.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:44.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 22018, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:43.437823+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:43.437823+0000', 'last_peered': '2024-09-11T05:24:43.437823+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:43.437823+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2386, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1133, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2385}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:44.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:44.559 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:44.559 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23700/6 objects degraded (395000.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":762,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1481080832,"bytes_avail":288429211648,"bytes_total":289910292480,"degraded_objects":23700,"degraded_total":6,"degraded_ratio":3950,"recovering_objects_per_sec":81,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:44.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:44.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:44.888 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:44.889 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:44.908 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-11T05:24:43.784340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18813,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003672"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295887+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295887+0000","last_peered":"2024-09-11T05:24:28.295887+0000","last_clean":"2024-09-11T05:24:28.295887+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295887+0000","last_undegraded":"2024-09-11T05:24:28.295887+0000","last_fullsized":"2024-09-11T05:24:28.295887+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":22018,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:43.437823+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:43.437823+0000","last_peered":"2024-09-11T05:24:43.437823+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:43.437823+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18812,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:47.910 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:48.237 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:48.237 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:48.251 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-11T05:24:47.785641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18813,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433888,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681632,"statfs":{"total":289910292480,"available":288441991168,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":22018,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:43.437823+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:43.437823+0000","last_peered":"2024-09-11T05:24:43.437823+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:43.437823+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18812,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:48.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:48.593 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:48.593 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:48.614 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-11T05:24:47.785641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18813,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433888,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681632,"statfs":{"total":289910292480,"available":288441991168,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":22018,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:43.437823+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:43.437823+0000","last_peered":"2024-09-11T05:24:43.437823+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:43.437823+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18812,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:48.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:48.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:48.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:48.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19955, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.287800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.287800+0000', 'last_peered': '2024-09-11T05:24:28.287800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.287800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:48.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:48.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 22018, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:43.437823+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:43.437823+0000', 'last_peered': '2024-09-11T05:24:43.437823+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:43.437823+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2386, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1133, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2385}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:48.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:48.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:48.993 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23700/6 objects degraded (395000.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":766,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1468301312,"bytes_avail":288441991168,"bytes_total":289910292480,"degraded_objects":23700,"degraded_total":6,"degraded_ratio":3950,"recovering_objects_per_sec":55,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:49.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:49.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:49.325 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:49.326 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:49.345 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-11T05:24:47.785641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18813,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433888,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681632,"statfs":{"total":289910292480,"available":288441991168,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":22018,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:43.437823+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:43.437823+0000","last_peered":"2024-09-11T05:24:43.437823+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:43.437823+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2385}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18812,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:52.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:52.684 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:52.685 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:52.705 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-11T05:24:51.787025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19178,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433888,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681632,"statfs":{"total":289910292480,"available":288441991168,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003802"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":22748,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:48.443359+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:48.443359+0000","last_peered":"2024-09-11T05:24:48.443359+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:48.443359+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1498,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19177,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215251,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:52.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:53.025 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:53.026 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:53.047 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-11T05:24:51.787025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19178,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433888,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681632,"statfs":{"total":289910292480,"available":288441991168,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003802"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20027,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289082+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289082+0000","last_peered":"2024-09-11T05:24:28.289082+0000","last_clean":"2024-09-11T05:24:28.289082+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289082+0000","last_undegraded":"2024-09-11T05:24:28.289082+0000","last_fullsized":"2024-09-11T05:24:28.289082+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":22748,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:48.443359+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:48.443359+0000","last_peered":"2024-09-11T05:24:48.443359+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:48.443359+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1498,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19177,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215251,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:53.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:53.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:53.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:53.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19955, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.287800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.287800+0000', 'last_peered': '2024-09-11T05:24:28.287800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.287800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:53.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:53.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 22748, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:48.443359+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:48.443359+0000', 'last_peered': '2024-09-11T05:24:48.443359+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:48.443359+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2021, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1498, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2020}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:53.050 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:53.427 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:53.427 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23335/6 objects degraded (388916.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":771,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1468301312,"bytes_avail":288441991168,"bytes_total":289910292480,"degraded_objects":23335,"degraded_total":6,"degraded_ratio":3889.1666666666661,"recovering_objects_per_sec":58,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":701,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:53.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:53.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:53.816 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:53.833 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-11T05:24:53.787640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19580,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438048,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677472,"statfs":{"total":289910292480,"available":288437731328,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003933"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":23552,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:53.448186+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:53.448186+0000","last_peered":"2024-09-11T05:24:53.448186+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:53.448186+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19579,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465328,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906512,"statfs":{"total":96636764160,"available":96160268288,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:56.835 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:57.158 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:57.158 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:57.176 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-11T05:24:55.788336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19580,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438048,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677472,"statfs":{"total":289910292480,"available":288437731328,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003996"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":23552,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:53.448186+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:53.448186+0000","last_peered":"2024-09-11T05:24:53.448186+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:53.448186+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19579,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465328,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906512,"statfs":{"total":96636764160,"available":96160268288,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:57.178 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:57.492 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:57.511 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-11T05:24:55.788336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19580,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438048,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677472,"statfs":{"total":289910292480,"available":288437731328,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003996"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":23552,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:53.448186+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:53.448186+0000","last_peered":"2024-09-11T05:24:53.448186+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:53.448186+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19579,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465328,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906512,"statfs":{"total":96636764160,"available":96160268288,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:24:57.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:24:57.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:57.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:24:57.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19955, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.287800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.287800+0000', 'last_peered': '2024-09-11T05:24:28.287800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.287800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:57.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:24:57.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 23552, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:53.448186+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:53.448186+0000', 'last_peered': '2024-09-11T05:24:53.448186+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:53.448186+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1619, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1900, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1618}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:24:57.514 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:24:57.925 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:57.925 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22933/6 objects degraded (382216.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":775,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1472561152,"bytes_avail":288437731328,"bytes_total":289910292480,"degraded_objects":22933,"degraded_total":6,"degraded_ratio":3822.1666666666661,"recovering_objects_per_sec":63,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":767,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:24:57.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:24:57.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:24:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:24:58.249 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:24:58.266 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-11T05:24:57.788901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19580,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442208,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673312,"statfs":{"total":289910292480,"available":288433471488,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003950"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":23552,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:53.448186+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:53.448186+0000","last_peered":"2024-09-11T05:24:53.448186+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:53.448186+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22933,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19579,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:01.267 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:01.583 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:01.584 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:01.602 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-11T05:24:59.789371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20039,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442208,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673312,"statfs":{"total":289910292480,"available":288433471488,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003729"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":24470,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:58.454218+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:58.454218+0000","last_peered":"2024-09-11T05:24:58.454218+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:58.454218+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20038,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215253,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:01.603 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:01.916 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:01.916 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:01.934 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-11T05:25:01.790069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20039,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442208,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673312,"statfs":{"total":289910292480,"available":288433471488,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":861,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003785"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":24470,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:58.454218+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:58.454218+0000","last_peered":"2024-09-11T05:24:58.454218+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:58.454218+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20038,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215253,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:01.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:01.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:01.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:01.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:01.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:01.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19955, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.287800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.287800+0000', 'last_peered': '2024-09-11T05:24:28.287800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.287800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:25:01.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 24470, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:58.454218+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:24:58.454218+0000', 'last_peered': '2024-09-11T05:24:58.454218+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:24:58.454218+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1160, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2359, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:01.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:02.325 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:02.325 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22474/6 objects degraded (374566.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":780,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1476820992,"bytes_avail":288433471488,"bytes_total":289910292480,"degraded_objects":22474,"degraded_total":6,"degraded_ratio":3745.6666666666661,"recovering_objects_per_sec":71,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":861,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:02.344 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:02.344 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:02.649 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:02.668 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-11T05:25:01.790069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20039,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442208,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673312,"statfs":{"total":289910292480,"available":288433471488,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":861,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003785"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":24470,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:58.454218+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:24:58.454218+0000","last_peered":"2024-09-11T05:24:58.454218+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:24:58.454218+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20038,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469488,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902352,"statfs":{"total":96636764160,"available":96156008448,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215253,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:05.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:05.999 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:06.000 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:06.018 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-11T05:25:05.791227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20556,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":25504,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:03.456011+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:25:03.456011+0000","last_peered":"2024-09-11T05:25:03.456011+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:25:03.456011+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2876,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20555,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215254,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:06.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:06.342 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:06.361 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-11T05:25:05.791227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20556,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":25504,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:03.456011+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:25:03.456011+0000","last_peered":"2024-09-11T05:25:03.456011+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:25:03.456011+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2876,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20555,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215254,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:06.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:06.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:06.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:06.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:06.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:06.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:06.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:06.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:06.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:06.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:06.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:06.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19955, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.287800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.287800+0000', 'last_peered': '2024-09-11T05:24:28.287800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.287800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:06.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:25:06.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 25504, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:03.456011+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:25:03.456011+0000', 'last_peered': '2024-09-11T05:25:03.456011+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:25:03.456011+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 643, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2876, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -642}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:06.365 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:06.734 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:06.734 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21957/6 objects degraded (365950.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":784,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1481080832,"bytes_avail":288429211648,"bytes_total":289910292480,"degraded_objects":21957,"degraded_total":6,"degraded_ratio":3659.5,"recovering_objects_per_sec":81,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:06.754 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:06.754 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:07.061 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:07.062 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:07.085 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-11T05:25:05.791227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20556,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446368,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281669152,"statfs":{"total":289910292480,"available":288429211648,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296125+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296125+0000","last_peered":"2024-09-11T05:24:28.296125+0000","last_clean":"2024-09-11T05:24:28.296125+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296125+0000","last_undegraded":"2024-09-11T05:24:28.296125+0000","last_fullsized":"2024-09-11T05:24:28.296125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":25504,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:03.456011+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:25:03.456011+0000","last_peered":"2024-09-11T05:25:03.456011+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:25:03.456011+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2876,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20555,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473648,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898192,"statfs":{"total":96636764160,"available":96151748608,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215254,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:10.087 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:10.438 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:10.439 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:10.459 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-11T05:25:09.908849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21175,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450528,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281664992,"statfs":{"total":289910292480,"available":288424951808,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.118779"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296177+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296177+0000","last_peered":"2024-09-11T05:24:28.296177+0000","last_clean":"2024-09-11T05:24:28.296177+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296177+0000","last_undegraded":"2024-09-11T05:24:28.296177+0000","last_fullsized":"2024-09-11T05:24:28.296177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":26742,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:08.451378+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:25:08.451378+0000","last_peered":"2024-09-11T05:25:08.451378+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:25:08.451378+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21174,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477808,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894032,"statfs":{"total":96636764160,"available":96147488768,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215255,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:10.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:10.827 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:10.848 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-11T05:25:09.908849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21175,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450528,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281664992,"statfs":{"total":289910292480,"available":288424951808,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.118779"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296177+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296177+0000","last_peered":"2024-09-11T05:24:28.296177+0000","last_clean":"2024-09-11T05:24:28.296177+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296177+0000","last_undegraded":"2024-09-11T05:24:28.296177+0000","last_fullsized":"2024-09-11T05:24:28.296177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":26742,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:08.451378+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:25:08.451378+0000","last_peered":"2024-09-11T05:25:08.451378+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:25:08.451378+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21174,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477808,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894032,"statfs":{"total":96636764160,"available":96147488768,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215255,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:10.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:10.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20288, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.290581+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:24:28.290581+0000', 'last_peered': '2024-09-11T05:24:28.290581+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:24:28.290581+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:10.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:10.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 20122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.292010+0000', 'last_change': '2024-09-11T05:14:14.294423+0000', 'last_active': '2024-09-11T05:24:28.292010+0000', 'last_peered': '2024-09-11T05:24:28.292010+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:24:28.292010+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:10.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:10.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19857, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288027+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:24:28.288027+0000', 'last_peered': '2024-09-11T05:24:28.288027+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:24:28.288027+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20295, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288764+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:24:28.288764+0000', 'last_peered': '2024-09-11T05:24:28.288764+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:24:28.288764+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19807, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.288759+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:24:28.288759+0000', 'last_peered': '2024-09-11T05:24:28.288759+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:24:28.288759+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19955, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:24:28.287800+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:24:28.287800+0000', 'last_peered': '2024-09-11T05:24:28.287800+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:24:28.287800+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T05:25:10.850 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19590", 'reported_seq': 26742, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:08.451378+0000', 'last_change': '2024-09-11T05:24:26.989531+0000', 'last_active': '2024-09-11T05:25:08.451378+0000', 'last_peered': '2024-09-11T05:25:08.451378+0000', 'last_clean': '2024-09-11T05:14:12.068296+0000', 'last_became_active': '2024-09-11T05:14:14.285519+0000', 'last_became_peered': '2024-09-11T05:14:14.285519+0000', 'last_unstale': '2024-09-11T05:25:08.451378+0000', 'last_undegraded': '2024-09-11T05:14:13.896878+0000', 'last_fullsized': '2024-09-11T05:14:13.746463+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3031", 'last_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:02.428529+0000', 'objects_scrubbed': 2982, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1183, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 24, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16597, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3495, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -23}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:10.851 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:11.293 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:11.294 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21338/6 objects degraded (355633.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":789,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1485340672,"bytes_avail":288424951808,"bytes_total":289910292480,"degraded_objects":21338,"degraded_total":6,"degraded_ratio":3556.3333333333339,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1136,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:11.317 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:11.317 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:11.668 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-11T05:25:09.908849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21175,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450528,"kb_used_data":171296,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281664992,"statfs":{"total":289910292480,"available":288424951808,"internally_reserved":0,"allocated":175407104,"data_stored":174473358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.118779"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20288,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.290581+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:24:28.290581+0000","last_peered":"2024-09-11T05:24:28.290581+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:24:28.290581+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20022,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292226+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:24:28.292226+0000","last_peered":"2024-09-11T05:24:28.292226+0000","last_clean":"2024-09-11T05:24:28.292226+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:24:28.292226+0000","last_undegraded":"2024-09-11T05:24:28.292226+0000","last_fullsized":"2024-09-11T05:24:28.292226+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26864,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295841+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:24:28.295841+0000","last_peered":"2024-09-11T05:24:28.295841+0000","last_clean":"2024-09-11T05:24:28.295841+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:24:28.295841+0000","last_undegraded":"2024-09-11T05:24:28.295841+0000","last_fullsized":"2024-09-11T05:24:28.295841+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20051,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292315+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:24:28.292315+0000","last_peered":"2024-09-11T05:24:28.292315+0000","last_clean":"2024-09-11T05:24:28.292315+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:24:28.292315+0000","last_undegraded":"2024-09-11T05:24:28.292315+0000","last_fullsized":"2024-09-11T05:24:28.292315+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20318,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286548+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:24:28.286548+0000","last_peered":"2024-09-11T05:24:28.286548+0000","last_clean":"2024-09-11T05:24:28.286548+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:24:28.286548+0000","last_undegraded":"2024-09-11T05:24:28.286548+0000","last_fullsized":"2024-09-11T05:24:28.286548+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27400,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295987+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:24:28.295987+0000","last_peered":"2024-09-11T05:24:28.295987+0000","last_clean":"2024-09-11T05:24:28.295987+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:24:28.295987+0000","last_undegraded":"2024-09-11T05:24:28.295987+0000","last_fullsized":"2024-09-11T05:24:28.295987+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22651,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296086+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:24:28.296086+0000","last_peered":"2024-09-11T05:24:28.296086+0000","last_clean":"2024-09-11T05:24:28.296086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:24:28.296086+0000","last_undegraded":"2024-09-11T05:24:28.296086+0000","last_fullsized":"2024-09-11T05:24:28.296086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20028,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.289150+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:24:28.289150+0000","last_peered":"2024-09-11T05:24:28.289150+0000","last_clean":"2024-09-11T05:24:28.289150+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:24:28.289150+0000","last_undegraded":"2024-09-11T05:24:28.289150+0000","last_fullsized":"2024-09-11T05:24:28.289150+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295566+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:24:28.295566+0000","last_peered":"2024-09-11T05:24:28.295566+0000","last_clean":"2024-09-11T05:24:28.295566+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:24:28.295566+0000","last_undegraded":"2024-09-11T05:24:28.295566+0000","last_fullsized":"2024-09-11T05:24:28.295566+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.292010+0000","last_change":"2024-09-11T05:14:14.294423+0000","last_active":"2024-09-11T05:24:28.292010+0000","last_peered":"2024-09-11T05:24:28.292010+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:24:28.292010+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292836+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:24:28.292836+0000","last_peered":"2024-09-11T05:24:28.292836+0000","last_clean":"2024-09-11T05:24:28.292836+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:24:28.292836+0000","last_undegraded":"2024-09-11T05:24:28.292836+0000","last_fullsized":"2024-09-11T05:24:28.292836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19857,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288027+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:24:28.288027+0000","last_peered":"2024-09-11T05:24:28.288027+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:24:28.288027+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.286281+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:24:28.286281+0000","last_peered":"2024-09-11T05:24:28.286281+0000","last_clean":"2024-09-11T05:24:28.286281+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:24:28.286281+0000","last_undegraded":"2024-09-11T05:24:28.286281+0000","last_fullsized":"2024-09-11T05:24:28.286281+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27226,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.358649+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:24:28.358649+0000","last_peered":"2024-09-11T05:24:28.358649+0000","last_clean":"2024-09-11T05:24:28.358649+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:24:28.358649+0000","last_undegraded":"2024-09-11T05:24:28.358649+0000","last_fullsized":"2024-09-11T05:24:28.358649+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295991+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:24:28.295991+0000","last_peered":"2024-09-11T05:24:28.295991+0000","last_clean":"2024-09-11T05:24:28.295991+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:24:28.295991+0000","last_undegraded":"2024-09-11T05:24:28.295991+0000","last_fullsized":"2024-09-11T05:24:28.295991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26813,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.292003+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:24:28.292003+0000","last_peered":"2024-09-11T05:24:28.292003+0000","last_clean":"2024-09-11T05:24:28.292003+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:24:28.292003+0000","last_undegraded":"2024-09-11T05:24:28.292003+0000","last_fullsized":"2024-09-11T05:24:28.292003+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20295,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288764+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:24:28.288764+0000","last_peered":"2024-09-11T05:24:28.288764+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:24:28.288764+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295835+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:24:28.295835+0000","last_peered":"2024-09-11T05:24:28.295835+0000","last_clean":"2024-09-11T05:24:28.295835+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:24:28.295835+0000","last_undegraded":"2024-09-11T05:24:28.295835+0000","last_fullsized":"2024-09-11T05:24:28.295835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19807,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.288759+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:24:28.288759+0000","last_peered":"2024-09-11T05:24:28.288759+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:24:28.288759+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295102+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:24:28.295102+0000","last_peered":"2024-09-11T05:24:28.295102+0000","last_clean":"2024-09-11T05:24:28.295102+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:24:28.295102+0000","last_undegraded":"2024-09-11T05:24:28.295102+0000","last_fullsized":"2024-09-11T05:24:28.295102+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19955,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:24:28.287800+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:24:28.287800+0000","last_peered":"2024-09-11T05:24:28.287800+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:24:28.287800+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.295952+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:24:28.295952+0000","last_peered":"2024-09-11T05:24:28.295952+0000","last_clean":"2024-09-11T05:24:28.295952+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:24:28.295952+0000","last_undegraded":"2024-09-11T05:24:28.295952+0000","last_fullsized":"2024-09-11T05:24:28.295952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":94,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T05:24:28.296177+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:24:28.296177+0000","last_peered":"2024-09-11T05:24:28.296177+0000","last_clean":"2024-09-11T05:24:28.296177+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:24:28.296177+0000","last_undegraded":"2024-09-11T05:24:28.296177+0000","last_fullsized":"2024-09-11T05:24:28.296177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":26742,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:08.451378+0000","last_change":"2024-09-11T05:24:26.989531+0000","last_active":"2024-09-11T05:25:08.451378+0000","last_peered":"2024-09-11T05:25:08.451378+0000","last_clean":"2024-09-11T05:14:12.068296+0000","last_became_active":"2024-09-11T05:14:14.285519+0000","last_became_peered":"2024-09-11T05:14:14.285519+0000","last_unstale":"2024-09-11T05:25:08.451378+0000","last_undegraded":"2024-09-11T05:14:13.896878+0000","last_fullsized":"2024-09-11T05:14:13.746463+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21174,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174071808,"data_stored":174026581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461140,"kb_used_data":114300,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910700,"statfs":{"total":96636764160,"available":96164556800,"internally_reserved":0,"allocated":117043200,"data_stored":116746266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477808,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894032,"statfs":{"total":96636764160,"available":96147488768,"internally_reserved":0,"allocated":491520,"data_stored":155719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215255,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511580,"kb_used_data":56516,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860260,"statfs":{"total":96636764160,"available":96112906240,"internally_reserved":0,"allocated":57872384,"data_stored":57571373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:14.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:15.044 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:15.045 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:15.064 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-11T05:25:13.793749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21753,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438088,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677432,"statfs":{"total":289910292480,"available":288437690368,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002521"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":21235,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:13.015851+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:13.015851+0000","last_peered":"2024-09-11T05:25:13.015851+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:13.015851+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21752,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:15.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:15.403 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:15.403 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:15.422 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-11T05:25:13.793749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21753,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438088,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677432,"statfs":{"total":289910292480,"available":288437690368,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002521"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":21235,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:13.015851+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:13.015851+0000","last_peered":"2024-09-11T05:25:13.015851+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:13.015851+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21752,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:15.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:15.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20292, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.916356+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:09.916356+0000', 'last_peered': '2024-09-11T05:25:09.916356+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:09.916356+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:15.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:15.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 21235, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:13.015851+0000', 'last_change': '2024-09-11T05:25:08.640823+0000', 'last_active': '2024-09-11T05:25:13.015851+0000', 'last_peered': '2024-09-11T05:25:13.015851+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:25:13.015851+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 554, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2973}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19861, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.913504+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:09.913504+0000', 'last_peered': '2024-09-11T05:25:09.913504+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:09.913504+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20299, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.915148+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:09.915148+0000', 'last_peered': '2024-09-11T05:25:09.915148+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:09.915148+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19811, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914202+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:09.914202+0000', 'last_peered': '2024-09-11T05:25:09.914202+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:09.914202+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:15.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19959, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914568+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:25:09.914568+0000', 'last_peered': '2024-09-11T05:25:09.914568+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:09.914568+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:15.425 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:15.917 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:15.917 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20760/6 objects degraded (346000.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":793,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1472602112,"bytes_avail":288437690368,"bytes_total":289910292480,"degraded_objects":20760,"degraded_total":6,"degraded_ratio":3460,"recovering_objects_per_sec":149,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1197,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:15.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:15.942 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:16.271 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:16.291 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-11T05:25:15.794357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21753,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438088,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677432,"statfs":{"total":289910292480,"available":288437690368,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002454"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":21235,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:13.015851+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:13.015851+0000","last_peered":"2024-09-11T05:25:13.015851+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:13.015851+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21752,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:19.294 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:19.640 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:19.641 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:19.658 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-11T05:25:17.794807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21753,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.901651"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":21235,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:13.015851+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:13.015851+0000","last_peered":"2024-09-11T05:25:13.015851+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:13.015851+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21752,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:19.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:19.978 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-11T05:25:19.795445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22487,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002880"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":22703,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:18.020930+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:18.020930+0000","last_peered":"2024-09-11T05:25:18.020930+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:18.020930+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22486,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215257,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:20.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:20.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20292, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.916356+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:09.916356+0000', 'last_peered': '2024-09-11T05:25:09.916356+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:09.916356+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:20.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:20.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 22703, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:18.020930+0000', 'last_change': '2024-09-11T05:25:08.640823+0000', 'last_active': '2024-09-11T05:25:18.020930+0000', 'last_peered': '2024-09-11T05:25:18.020930+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:25:18.020930+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2239, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1288, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19861, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.913504+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:09.913504+0000', 'last_peered': '2024-09-11T05:25:09.913504+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:09.913504+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20299, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.915148+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:09.915148+0000', 'last_peered': '2024-09-11T05:25:09.915148+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:09.915148+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19811, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914202+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:09.914202+0000', 'last_peered': '2024-09-11T05:25:09.914202+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:09.914202+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:20.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19959, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914568+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:25:09.914568+0000', 'last_peered': '2024-09-11T05:25:09.914568+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:09.914568+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:20.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:20.376 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:20.376 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20026/6 objects degraded (333766.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":798,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1476857856,"bytes_avail":288433434624,"bytes_total":289910292480,"degraded_objects":20026,"degraded_total":6,"degraded_ratio":3337.6666666666661,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1312,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:20.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:20.396 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:20.709 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:20.728 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-11T05:25:19.795445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22487,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002880"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":22703,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:18.020930+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:18.020930+0000","last_peered":"2024-09-11T05:25:18.020930+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:18.020930+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22486,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215257,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:23.730 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:24.096 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:24.116 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-11T05:25:23.796483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22971,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450564,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281664956,"statfs":{"total":289910292480,"available":288424914944,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003292"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":23671,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:23.024644+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:23.024644+0000","last_peered":"2024-09-11T05:25:23.024644+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:23.024644+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22970,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477816,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894024,"statfs":{"total":96636764160,"available":96147480576,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:24.118 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:24.469 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:24.469 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:24.490 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-11T05:25:23.796483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22971,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450564,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281664956,"statfs":{"total":289910292480,"available":288424914944,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003292"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":23671,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:23.024644+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:23.024644+0000","last_peered":"2024-09-11T05:25:23.024644+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:23.024644+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22970,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477816,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894024,"statfs":{"total":96636764160,"available":96147480576,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:24.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:24.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20292, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.916356+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:09.916356+0000', 'last_peered': '2024-09-11T05:25:09.916356+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:09.916356+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:24.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:24.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 23671, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:23.024644+0000', 'last_change': '2024-09-11T05:25:08.640823+0000', 'last_active': '2024-09-11T05:25:23.024644+0000', 'last_peered': '2024-09-11T05:25:23.024644+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:25:23.024644+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1755, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1772, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:24.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:24.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19861, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.913504+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:09.913504+0000', 'last_peered': '2024-09-11T05:25:09.913504+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:09.913504+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:24.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:24.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20299, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.915148+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:09.915148+0000', 'last_peered': '2024-09-11T05:25:09.915148+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:09.915148+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:24.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:24.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19811, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914202+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:09.914202+0000', 'last_peered': '2024-09-11T05:25:09.914202+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:09.914202+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:24.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:24.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19959, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914568+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:25:09.914568+0000', 'last_peered': '2024-09-11T05:25:09.914568+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:09.914568+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:24.493 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:24.976 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:24.976 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19542/6 objects degraded (325700.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":802,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1485377536,"bytes_avail":288424914944,"bytes_total":289910292480,"degraded_objects":19542,"degraded_total":6,"degraded_ratio":3257,"recovering_objects_per_sec":149,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:24.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:24.996 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:25.298 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:25.299 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:25.318 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-11T05:25:23.796483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22971,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450564,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1279155,"kb_avail":281664956,"statfs":{"total":289910292480,"available":288424914944,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1309854721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003292"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":23671,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:23.024644+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:23.024644+0000","last_peered":"2024-09-11T05:25:23.024644+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:23.024644+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22970,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477816,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93894024,"statfs":{"total":96636764160,"available":96147480576,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":488763051},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:28.320 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:28.826 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:28.827 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:28.847 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-11T05:25:27.797900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22971,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433924,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681596,"statfs":{"total":289910292480,"available":288441954304,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003542"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":23671,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:23.024644+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:23.024644+0000","last_peered":"2024-09-11T05:25:23.024644+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:23.024644+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22970,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461176,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910664,"statfs":{"total":96636764160,"available":96164519936,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:28.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:29.165 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:29.185 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-11T05:25:27.797900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22971,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433924,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681596,"statfs":{"total":289910292480,"available":288441954304,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003542"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":23671,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:23.024644+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:23.024644+0000","last_peered":"2024-09-11T05:25:23.024644+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:23.024644+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1755}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913221+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913221+0000","last_peered":"2024-09-11T05:25:09.913221+0000","last_clean":"2024-09-11T05:25:09.913221+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913221+0000","last_undegraded":"2024-09-11T05:25:09.913221+0000","last_fullsized":"2024-09-11T05:25:09.913221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22970,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461176,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910664,"statfs":{"total":96636764160,"available":96164519936,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:29.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:29.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20292, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.916356+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:09.916356+0000', 'last_peered': '2024-09-11T05:25:09.916356+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:09.916356+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:29.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:29.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 23671, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:23.024644+0000', 'last_change': '2024-09-11T05:25:08.640823+0000', 'last_active': '2024-09-11T05:25:23.024644+0000', 'last_peered': '2024-09-11T05:25:23.024644+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:25:23.024644+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1755, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1772, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19861, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.913504+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:09.913504+0000', 'last_peered': '2024-09-11T05:25:09.913504+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:09.913504+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20299, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.915148+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:09.915148+0000', 'last_peered': '2024-09-11T05:25:09.915148+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:09.915148+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19811, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914202+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:09.914202+0000', 'last_peered': '2024-09-11T05:25:09.914202+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:09.914202+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:29.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19959, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914568+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:25:09.914568+0000', 'last_peered': '2024-09-11T05:25:09.914568+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:09.914568+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:29.188 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:29.559 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:29.559 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19542/6 objects degraded (325700.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":807,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1468338176,"bytes_avail":288441954304,"bytes_total":289910292480,"degraded_objects":19542,"degraded_total":6,"degraded_ratio":3257,"recovering_objects_per_sec":101,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1218,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:29.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:29.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:29.933 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:29.934 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:29.953 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-11T05:25:29.798518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23382,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433924,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681596,"statfs":{"total":289910292480,"available":288441954304,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1629,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003710"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":24493,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:28.022917+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:28.022917+0000","last_peered":"2024-09-11T05:25:28.022917+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:28.022917+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23381,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461176,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910664,"statfs":{"total":96636764160,"available":96164519936,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215259,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:32.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:33.268 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:33.269 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:33.287 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-11T05:25:31.798991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23382,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433924,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681596,"statfs":{"total":289910292480,"available":288441954304,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":895,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003546"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":24493,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:28.022917+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:28.022917+0000","last_peered":"2024-09-11T05:25:28.022917+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:28.022917+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23381,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461176,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910664,"statfs":{"total":96636764160,"available":96164519936,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215259,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:33.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:33.627 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:33.645 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-11T05:25:31.798991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23382,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1433924,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281681596,"statfs":{"total":289910292480,"available":288441954304,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-895,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":895,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003546"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":24493,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:28.022917+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:28.022917+0000","last_peered":"2024-09-11T05:25:28.022917+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:28.022917+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23381,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461176,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910664,"statfs":{"total":96636764160,"available":96164519936,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471723691},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215259,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:33.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20292, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.916356+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:09.916356+0000', 'last_peered': '2024-09-11T05:25:09.916356+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:09.916356+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 24493, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:28.022917+0000', 'last_change': '2024-09-11T05:25:08.640823+0000', 'last_active': '2024-09-11T05:25:28.022917+0000', 'last_peered': '2024-09-11T05:25:28.022917+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:25:28.022917+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1344, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2183, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1344}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19861, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.913504+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:09.913504+0000', 'last_peered': '2024-09-11T05:25:09.913504+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:09.913504+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20299, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.915148+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:09.915148+0000', 'last_peered': '2024-09-11T05:25:09.915148+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:09.915148+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:33.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19811, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914202+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:09.914202+0000', 'last_peered': '2024-09-11T05:25:09.914202+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:09.914202+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:33.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:33.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19959, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914568+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:25:09.914568+0000', 'last_peered': '2024-09-11T05:25:09.914568+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:09.914568+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:33.648 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:34.033 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:34.034 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18667/6 objects degraded (311116.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":811,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1472598016,"bytes_avail":288437694464,"bytes_total":289910292480,"degraded_objects":18667,"degraded_total":6,"degraded_ratio":3111.1666666666661,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:34.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:34.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:34.374 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:34.375 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:34.393 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-11T05:25:33.799560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23846,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438084,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677436,"statfs":{"total":289910292480,"available":288437694464,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003703"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":25421,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:33.025725+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:33.025725+0000","last_peered":"2024-09-11T05:25:33.025725+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:33.025725+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-880}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23845,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465336,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906504,"statfs":{"total":96636764160,"available":96160260096,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215260,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:37.395 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:37.720 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:37.740 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-11T05:25:35.799997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23846,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438084,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281677436,"statfs":{"total":289910292480,"available":288437694464,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1297075201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003514"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":25421,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:33.025725+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:33.025725+0000","last_peered":"2024-09-11T05:25:33.025725+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:33.025725+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-880}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23845,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465336,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906504,"statfs":{"total":96636764160,"available":96160260096,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475983531},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215260,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:37.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:38.061 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:38.062 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:38.081 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-11T05:25:37.800621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23846,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":25421,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:33.025725+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:33.025725+0000","last_peered":"2024-09-11T05:25:33.025725+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:33.025725+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-880}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23845,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215260,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20292, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.916356+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:09.916356+0000', 'last_peered': '2024-09-11T05:25:09.916356+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:09.916356+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 25421, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:33.025725+0000', 'last_change': '2024-09-11T05:25:08.640823+0000', 'last_active': '2024-09-11T05:25:33.025725+0000', 'last_peered': '2024-09-11T05:25:33.025725+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:25:33.025725+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 880, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2647, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -880}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19861, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.913504+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:09.913504+0000', 'last_peered': '2024-09-11T05:25:09.913504+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:09.913504+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20299, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.915148+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:09.915148+0000', 'last_peered': '2024-09-11T05:25:09.915148+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:09.915148+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:38.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19811, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914202+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:09.914202+0000', 'last_peered': '2024-09-11T05:25:09.914202+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:09.914202+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:38.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:38.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19959, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914568+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:25:09.914568+0000', 'last_peered': '2024-09-11T05:25:09.914568+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:09.914568+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:38.084 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:38.491 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:38.491 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18667/6 objects degraded (311116.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":816,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1476857856,"bytes_avail":288433434624,"bytes_total":289910292480,"degraded_objects":18667,"degraded_total":6,"degraded_ratio":3111.1666666666661,"recovering_objects_per_sec":72,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":875,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:38.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:38.511 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:38.825 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:38.843 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-11T05:25:37.800621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23846,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":25421,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:33.025725+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:33.025725+0000","last_peered":"2024-09-11T05:25:33.025725+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:33.025725+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-880}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23845,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215260,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:41.845 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:42.212 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:42.212 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:42.235 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-11T05:25:41.801818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24444,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002258"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":26617,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:38.028927+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:38.028927+0000","last_peered":"2024-09-11T05:25:38.028927+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:38.028927+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-282}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24443,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215261,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:42.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:42.580 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:42.580 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:42.600 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-11T05:25:41.801818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24444,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002258"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":26617,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:38.028927+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:38.028927+0000","last_peered":"2024-09-11T05:25:38.028927+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:38.028927+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-282}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24443,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215261,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20292, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.916356+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:09.916356+0000', 'last_peered': '2024-09-11T05:25:09.916356+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:09.916356+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'19971", 'reported_seq': 26617, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:38.028927+0000', 'last_change': '2024-09-11T05:25:08.640823+0000', 'last_active': '2024-09-11T05:25:38.028927+0000', 'last_peered': '2024-09-11T05:25:38.028927+0000', 'last_clean': '2024-09-11T05:14:12.102528+0000', 'last_became_active': '2024-09-11T05:14:14.286206+0000', 'last_became_peered': '2024-09-11T05:14:14.286206+0000', 'last_unstale': '2024-09-11T05:25:38.028927+0000', 'last_undegraded': '2024-09-11T05:14:13.823022+0000', 'last_fullsized': '2024-09-11T05:14:13.737719+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1981", 'last_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:13:49.372780+0000', 'objects_scrubbed': 1925, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1099, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 282, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18039, 'num_write_kb': 32228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3245, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -282}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19861, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.913504+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:09.913504+0000', 'last_peered': '2024-09-11T05:25:09.913504+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:09.913504+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20299, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.915148+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:09.915148+0000', 'last_peered': '2024-09-11T05:25:09.915148+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:09.915148+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:42.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:42.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19811, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914202+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:09.914202+0000', 'last_peered': '2024-09-11T05:25:09.914202+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:09.914202+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:42.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:42.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 19959, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:09.914568+0000', 'last_change': '2024-09-11T05:14:14.294570+0000', 'last_active': '2024-09-11T05:25:09.914568+0000', 'last_peered': '2024-09-11T05:25:09.914568+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:09.914568+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:42.603 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:43.038 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:43.039 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18069/6 objects degraded (301150.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":820,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1476857856,"bytes_avail":288433434624,"bytes_total":289910292480,"degraded_objects":18069,"degraded_total":6,"degraded_ratio":3011.5,"recovering_objects_per_sec":74,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":598,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:43.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:43.058 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:43.717 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:43.735 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-11T05:25:41.801818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24444,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442244,"kb_used_data":171336,"kb_used_omap":12,"kb_used_meta":1270835,"kb_avail":281673276,"statfs":{"total":289910292480,"available":288433434624,"internally_reserved":0,"allocated":175448064,"data_stored":174489341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1301335041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002258"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20292,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.916356+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:09.916356+0000","last_peered":"2024-09-11T05:25:09.916356+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:09.916356+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20026,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934953+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:09.934953+0000","last_peered":"2024-09-11T05:25:09.934953+0000","last_clean":"2024-09-11T05:25:09.934953+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:09.934953+0000","last_undegraded":"2024-09-11T05:25:09.934953+0000","last_fullsized":"2024-09-11T05:25:09.934953+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26868,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921016+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:09.921016+0000","last_peered":"2024-09-11T05:25:09.921016+0000","last_clean":"2024-09-11T05:25:09.921016+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:09.921016+0000","last_undegraded":"2024-09-11T05:25:09.921016+0000","last_fullsized":"2024-09-11T05:25:09.921016+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20055,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934769+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:09.934769+0000","last_peered":"2024-09-11T05:25:09.934769+0000","last_clean":"2024-09-11T05:25:09.934769+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:09.934769+0000","last_undegraded":"2024-09-11T05:25:09.934769+0000","last_fullsized":"2024-09-11T05:25:09.934769+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20322,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913813+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:09.913813+0000","last_peered":"2024-09-11T05:25:09.913813+0000","last_clean":"2024-09-11T05:25:09.913813+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:09.913813+0000","last_undegraded":"2024-09-11T05:25:09.913813+0000","last_fullsized":"2024-09-11T05:25:09.913813+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27404,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921106+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:09.921106+0000","last_peered":"2024-09-11T05:25:09.921106+0000","last_clean":"2024-09-11T05:25:09.921106+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:09.921106+0000","last_undegraded":"2024-09-11T05:25:09.921106+0000","last_fullsized":"2024-09-11T05:25:09.921106+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22655,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921015+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:09.921015+0000","last_peered":"2024-09-11T05:25:09.921015+0000","last_clean":"2024-09-11T05:25:09.921015+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:09.921015+0000","last_undegraded":"2024-09-11T05:25:09.921015+0000","last_fullsized":"2024-09-11T05:25:09.921015+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20032,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.912872+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:09.912872+0000","last_peered":"2024-09-11T05:25:09.912872+0000","last_clean":"2024-09-11T05:25:09.912872+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:09.912872+0000","last_undegraded":"2024-09-11T05:25:09.912872+0000","last_fullsized":"2024-09-11T05:25:09.912872+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.920841+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:09.920841+0000","last_peered":"2024-09-11T05:25:09.920841+0000","last_clean":"2024-09-11T05:25:09.920841+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:09.920841+0000","last_undegraded":"2024-09-11T05:25:09.920841+0000","last_fullsized":"2024-09-11T05:25:09.920841+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":26617,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:38.028927+0000","last_change":"2024-09-11T05:25:08.640823+0000","last_active":"2024-09-11T05:25:38.028927+0000","last_peered":"2024-09-11T05:25:38.028927+0000","last_clean":"2024-09-11T05:14:12.102528+0000","last_became_active":"2024-09-11T05:14:14.286206+0000","last_became_peered":"2024-09-11T05:14:14.286206+0000","last_unstale":"2024-09-11T05:25:38.028927+0000","last_undegraded":"2024-09-11T05:14:13.823022+0000","last_fullsized":"2024-09-11T05:14:13.737719+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-282}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934297+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:09.934297+0000","last_peered":"2024-09-11T05:25:09.934297+0000","last_clean":"2024-09-11T05:25:09.934297+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:09.934297+0000","last_undegraded":"2024-09-11T05:25:09.934297+0000","last_fullsized":"2024-09-11T05:25:09.934297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19861,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.913504+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:09.913504+0000","last_peered":"2024-09-11T05:25:09.913504+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:09.913504+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":130,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.913305+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:09.913305+0000","last_peered":"2024-09-11T05:25:09.913305+0000","last_clean":"2024-09-11T05:25:09.913305+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:09.913305+0000","last_undegraded":"2024-09-11T05:25:09.913305+0000","last_fullsized":"2024-09-11T05:25:09.913305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934441+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:09.934441+0000","last_peered":"2024-09-11T05:25:09.934441+0000","last_clean":"2024-09-11T05:25:09.934441+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:09.934441+0000","last_undegraded":"2024-09-11T05:25:09.934441+0000","last_fullsized":"2024-09-11T05:25:09.934441+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.922293+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:09.922293+0000","last_peered":"2024-09-11T05:25:09.922293+0000","last_clean":"2024-09-11T05:25:09.922293+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:09.922293+0000","last_undegraded":"2024-09-11T05:25:09.922293+0000","last_fullsized":"2024-09-11T05:25:09.922293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26817,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.934219+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:09.934219+0000","last_peered":"2024-09-11T05:25:09.934219+0000","last_clean":"2024-09-11T05:25:09.934219+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:09.934219+0000","last_undegraded":"2024-09-11T05:25:09.934219+0000","last_fullsized":"2024-09-11T05:25:09.934219+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20299,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.915148+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:09.915148+0000","last_peered":"2024-09-11T05:25:09.915148+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:09.915148+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921863+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:09.921863+0000","last_peered":"2024-09-11T05:25:09.921863+0000","last_clean":"2024-09-11T05:25:09.921863+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:09.921863+0000","last_undegraded":"2024-09-11T05:25:09.921863+0000","last_fullsized":"2024-09-11T05:25:09.921863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914202+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:09.914202+0000","last_peered":"2024-09-11T05:25:09.914202+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:09.914202+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.935099+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:09.935099+0000","last_peered":"2024-09-11T05:25:09.935099+0000","last_clean":"2024-09-11T05:25:09.935099+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:09.935099+0000","last_undegraded":"2024-09-11T05:25:09.935099+0000","last_fullsized":"2024-09-11T05:25:09.935099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:09.914568+0000","last_change":"2024-09-11T05:14:14.294570+0000","last_active":"2024-09-11T05:25:09.914568+0000","last_peered":"2024-09-11T05:25:09.914568+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:09.914568+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921199+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:09.921199+0000","last_peered":"2024-09-11T05:25:09.921199+0000","last_clean":"2024-09-11T05:25:09.921199+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:09.921199+0000","last_undegraded":"2024-09-11T05:25:09.921199+0000","last_fullsized":"2024-09-11T05:25:09.921199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:09.921397+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:09.921397+0000","last_peered":"2024-09-11T05:25:09.921397+0000","last_clean":"2024-09-11T05:25:09.921397+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:09.921397+0000","last_undegraded":"2024-09-11T05:25:09.921397+0000","last_fullsized":"2024-09-11T05:25:09.921397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19755,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T05:25:10.045089+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:10.045089+0000","last_peered":"2024-09-11T05:25:10.045089+0000","last_clean":"2024-09-11T05:25:10.045089+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:10.045089+0000","last_undegraded":"2024-09-11T05:25:10.045089+0000","last_fullsized":"2024-09-11T05:25:10.045089+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24443,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174063616,"data_stored":174025347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":114316,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":117059584,"data_stored":116752005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":160224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480243371},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215261,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511592,"kb_used_data":56532,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860248,"statfs":{"total":96636764160,"available":96112893952,"internally_reserved":0,"allocated":57888768,"data_stored":57577112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116600832,"data_stored":116597353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:46.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:47.235 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:47.235 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:47.254 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-11T05:25:45.802698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25180,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403680,"kb_used_data":128608,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281711840,"statfs":{"total":289910292480,"available":288472924160,"internally_reserved":0,"allocated":131694592,"data_stored":130719579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002076"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373531+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373531+0000","last_peered":"2024-09-11T05:25:41.373531+0000","last_clean":"2024-09-11T05:25:41.373531+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373531+0000","last_undegraded":"2024-09-11T05:25:41.373531+0000","last_fullsized":"2024-09-11T05:25:41.373531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27234,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373675+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373675+0000","last_peered":"2024-09-11T05:25:41.373675+0000","last_clean":"2024-09-11T05:25:41.373675+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373675+0000","last_undegraded":"2024-09-11T05:25:41.373675+0000","last_fullsized":"2024-09-11T05:25:41.373675+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26821,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374056+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374056+0000","last_peered":"2024-09-11T05:25:41.374056+0000","last_clean":"2024-09-11T05:25:41.374056+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374056+0000","last_undegraded":"2024-09-11T05:25:41.374056+0000","last_fullsized":"2024-09-11T05:25:41.374056+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374610+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374610+0000","last_peered":"2024-09-11T05:25:41.374610+0000","last_clean":"2024-09-11T05:25:41.374610+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374610+0000","last_undegraded":"2024-09-11T05:25:41.374610+0000","last_fullsized":"2024-09-11T05:25:41.374610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":20872,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:43.458254+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:43.458254+0000","last_peered":"2024-09-11T05:25:43.458254+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:43.458254+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":454,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3092}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25179,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":499712,"data_stored":161845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215262,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:47.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:47.605 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:47.606 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:47.626 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-11T05:25:45.802698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25180,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403680,"kb_used_data":128608,"kb_used_omap":12,"kb_used_meta":1274995,"kb_avail":281711840,"statfs":{"total":289910292480,"available":288472924160,"internally_reserved":0,"allocated":131694592,"data_stored":130719579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1305594881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002076"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373531+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373531+0000","last_peered":"2024-09-11T05:25:41.373531+0000","last_clean":"2024-09-11T05:25:41.373531+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373531+0000","last_undegraded":"2024-09-11T05:25:41.373531+0000","last_fullsized":"2024-09-11T05:25:41.373531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27234,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373675+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373675+0000","last_peered":"2024-09-11T05:25:41.373675+0000","last_clean":"2024-09-11T05:25:41.373675+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373675+0000","last_undegraded":"2024-09-11T05:25:41.373675+0000","last_fullsized":"2024-09-11T05:25:41.373675+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26821,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374056+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374056+0000","last_peered":"2024-09-11T05:25:41.374056+0000","last_clean":"2024-09-11T05:25:41.374056+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374056+0000","last_undegraded":"2024-09-11T05:25:41.374056+0000","last_fullsized":"2024-09-11T05:25:41.374056+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374610+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374610+0000","last_peered":"2024-09-11T05:25:41.374610+0000","last_clean":"2024-09-11T05:25:41.374610+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374610+0000","last_undegraded":"2024-09-11T05:25:41.374610+0000","last_fullsized":"2024-09-11T05:25:41.374610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":20872,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:43.458254+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:43.458254+0000","last_peered":"2024-09-11T05:25:43.458254+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:43.458254+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":454,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3092}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25179,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":499712,"data_stored":161845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484503211},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215262,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:47.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:47.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20296, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.367899+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:41.367899+0000', 'last_peered': '2024-09-11T05:25:41.367899+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:41.367899+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:47.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:47.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19865, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.360632+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:41.360632+0000', 'last_peered': '2024-09-11T05:25:41.360632+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:41.360632+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:47.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:47.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20303, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364747+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:41.364747+0000', 'last_peered': '2024-09-11T05:25:41.364747+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:41.364747+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:47.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:47.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19815, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364902+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:41.364902+0000', 'last_peered': '2024-09-11T05:25:41.364902+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:41.364902+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:47.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:47.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 20872, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:43.458254+0000', 'last_change': '2024-09-11T05:25:39.846459+0000', 'last_active': '2024-09-11T05:25:43.458254+0000', 'last_peered': '2024-09-11T05:25:43.458254+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:43.458254+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3092, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 454, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3092}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:47.628 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:48.050 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:48.050 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17333/6 objects degraded (288883.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":825,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1437368320,"bytes_avail":288472924160,"bytes_total":289910292480,"degraded_objects":17333,"degraded_total":6,"degraded_ratio":2888.8333333333339,"recovering_objects_per_sec":166,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1334,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:48.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:48.070 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:48.407 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:48.408 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:48.429 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-11T05:25:47.803282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25180,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413412,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281702108,"statfs":{"total":289910292480,"available":288462958592,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-736,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":736,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002159"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373531+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373531+0000","last_peered":"2024-09-11T05:25:41.373531+0000","last_clean":"2024-09-11T05:25:41.373531+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373531+0000","last_undegraded":"2024-09-11T05:25:41.373531+0000","last_fullsized":"2024-09-11T05:25:41.373531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27234,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373675+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373675+0000","last_peered":"2024-09-11T05:25:41.373675+0000","last_clean":"2024-09-11T05:25:41.373675+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373675+0000","last_undegraded":"2024-09-11T05:25:41.373675+0000","last_fullsized":"2024-09-11T05:25:41.373675+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26821,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374056+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374056+0000","last_peered":"2024-09-11T05:25:41.374056+0000","last_clean":"2024-09-11T05:25:41.374056+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374056+0000","last_undegraded":"2024-09-11T05:25:41.374056+0000","last_fullsized":"2024-09-11T05:25:41.374056+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374610+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374610+0000","last_peered":"2024-09-11T05:25:41.374610+0000","last_clean":"2024-09-11T05:25:41.374610+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374610+0000","last_undegraded":"2024-09-11T05:25:41.374610+0000","last_fullsized":"2024-09-11T05:25:41.374610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":20872,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:43.458254+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:43.458254+0000","last_peered":"2024-09-11T05:25:43.458254+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:43.458254+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":454,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3092}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25179,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215262,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:51.432 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:51.765 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:51.766 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:51.786 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-09-11T05:25:49.803642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25926,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413412,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281702108,"statfs":{"total":289910292480,"available":288462958592,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.449801"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26821,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374056+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374056+0000","last_peered":"2024-09-11T05:25:41.374056+0000","last_clean":"2024-09-11T05:25:41.374056+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374056+0000","last_undegraded":"2024-09-11T05:25:41.374056+0000","last_fullsized":"2024-09-11T05:25:41.374056+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":22364,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:48.462046+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:48.462046+0000","last_peered":"2024-09-11T05:25:48.462046+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:48.462046+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25925,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215263,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:51.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:52.135 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:52.135 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:52.154 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-11T05:25:51.804054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25926,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413412,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281702108,"statfs":{"total":289910292480,"available":288462958592,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.448019"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26821,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374056+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374056+0000","last_peered":"2024-09-11T05:25:41.374056+0000","last_clean":"2024-09-11T05:25:41.374056+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374056+0000","last_undegraded":"2024-09-11T05:25:41.374056+0000","last_fullsized":"2024-09-11T05:25:41.374056+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":22364,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:48.462046+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:48.462046+0000","last_peered":"2024-09-11T05:25:48.462046+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:48.462046+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25925,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215263,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:52.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:52.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20296, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.367899+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:41.367899+0000', 'last_peered': '2024-09-11T05:25:41.367899+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:41.367899+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:52.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:52.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19865, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.360632+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:41.360632+0000', 'last_peered': '2024-09-11T05:25:41.360632+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:41.360632+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:52.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:52.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20303, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364747+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:41.364747+0000', 'last_peered': '2024-09-11T05:25:41.364747+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:41.364747+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:52.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:52.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19815, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364902+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:41.364902+0000', 'last_peered': '2024-09-11T05:25:41.364902+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:41.364902+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:52.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:52.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 22364, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:48.462046+0000', 'last_change': '2024-09-11T05:25:39.846459+0000', 'last_active': '2024-09-11T05:25:48.462046+0000', 'last_peered': '2024-09-11T05:25:48.462046+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:48.462046+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2346, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1200, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2346}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:52.157 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:52.533 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:52.533 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16587/6 objects degraded (276450.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":830,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1447333888,"bytes_avail":288462958592,"bytes_total":289910292480,"degraded_objects":16587,"degraded_total":6,"degraded_ratio":2764.5,"recovering_objects_per_sec":141,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1482,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:52.557 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:52.557 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:52.895 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:52.896 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:52.913 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-11T05:25:51.804054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25926,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413412,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281702108,"statfs":{"total":289910292480,"available":288462958592,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.448019"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26821,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374056+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374056+0000","last_peered":"2024-09-11T05:25:41.374056+0000","last_clean":"2024-09-11T05:25:41.374056+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374056+0000","last_undegraded":"2024-09-11T05:25:41.374056+0000","last_fullsized":"2024-09-11T05:25:41.374056+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":22364,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:48.462046+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:48.462046+0000","last_peered":"2024-09-11T05:25:48.462046+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:48.462046+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25925,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215263,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:55.915 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:56.257 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:56.258 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:56.276 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-11T05:25:55.805413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26471,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417572,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281697948,"statfs":{"total":289910292480,"available":288458698752,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003192"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":23454,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:53.455225+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:53.455225+0000","last_peered":"2024-09-11T05:25:53.455225+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:53.455225+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1801,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26470,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487552,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884288,"statfs":{"total":96636764160,"available":96137510912,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:56.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:56.599 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:56.599 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:56.617 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-11T05:25:55.805413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26471,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417572,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281697948,"statfs":{"total":289910292480,"available":288458698752,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003192"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":23454,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:53.455225+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:53.455225+0000","last_peered":"2024-09-11T05:25:53.455225+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:53.455225+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1801,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26470,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487552,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884288,"statfs":{"total":96636764160,"available":96137510912,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20296, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.367899+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:41.367899+0000', 'last_peered': '2024-09-11T05:25:41.367899+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:41.367899+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19865, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.360632+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:41.360632+0000', 'last_peered': '2024-09-11T05:25:41.360632+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:41.360632+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20303, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364747+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:41.364747+0000', 'last_peered': '2024-09-11T05:25:41.364747+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:41.364747+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19815, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364902+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:41.364902+0000', 'last_peered': '2024-09-11T05:25:41.364902+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:41.364902+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:25:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 23454, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:53.455225+0000', 'last_change': '2024-09-11T05:25:39.846459+0000', 'last_active': '2024-09-11T05:25:53.455225+0000', 'last_peered': '2024-09-11T05:25:53.455225+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:53.455225+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1801, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1745, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1801}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:25:56.620 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:25:57.005 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:57.005 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16042/6 objects degraded (267366.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":834,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1451593728,"bytes_avail":288458698752,"bytes_total":289910292480,"degraded_objects":16042,"degraded_total":6,"degraded_ratio":2673.6666666666661,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1291,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:25:57.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:25:57.025 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:25:57.333 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:25:57.333 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:25:57.352 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-11T05:25:55.805413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26471,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417572,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281697948,"statfs":{"total":289910292480,"available":288458698752,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003192"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19815,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364902+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.364902+0000","last_peered":"2024-09-11T05:25:41.364902+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.364902+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":23454,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:53.455225+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:53.455225+0000","last_peered":"2024-09-11T05:25:53.455225+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:53.455225+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1801,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26470,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487552,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884288,"statfs":{"total":96636764160,"available":96137510912,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:00.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:00.689 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:00.689 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:00.708 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-11T05:25:59.806629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26889,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421732,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281693788,"statfs":{"total":289910292480,"available":288454438912,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1709,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003346"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":24290,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:58.460613+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:58.460613+0000","last_peered":"2024-09-11T05:25:58.460613+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:58.460613+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26888,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491712,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880128,"statfs":{"total":96636764160,"available":96133251072,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215265,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:00.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:01.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:01.026 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:01.045 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-11T05:25:59.806629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26889,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421732,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281693788,"statfs":{"total":289910292480,"available":288454438912,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1709,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003346"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":24290,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:58.460613+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:58.460613+0000","last_peered":"2024-09-11T05:25:58.460613+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:58.460613+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26888,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491712,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880128,"statfs":{"total":96636764160,"available":96133251072,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215265,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:01.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:01.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20296, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.367899+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:41.367899+0000', 'last_peered': '2024-09-11T05:25:41.367899+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:41.367899+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:01.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:01.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19865, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.360632+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:41.360632+0000', 'last_peered': '2024-09-11T05:25:41.360632+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:41.360632+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:01.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:01.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20303, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364747+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:41.364747+0000', 'last_peered': '2024-09-11T05:25:41.364747+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:41.364747+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:01.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:01.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19816, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.366260+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:41.366260+0000', 'last_peered': '2024-09-11T05:25:41.366260+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:41.366260+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:01.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:26:01.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 24290, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:58.460613+0000', 'last_change': '2024-09-11T05:25:39.846459+0000', 'last_active': '2024-09-11T05:25:58.460613+0000', 'last_peered': '2024-09-11T05:25:58.460613+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:25:58.460613+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1383, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2163, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1383}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:01.047 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:01.444 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:01.444 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15624/6 objects degraded (260400.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":839,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1455853568,"bytes_avail":288454438912,"bytes_total":289910292480,"degraded_objects":15624,"degraded_total":6,"degraded_ratio":2604,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1709,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:01.464 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:01.464 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:01.804 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:01.805 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:01.824 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-11T05:25:59.806629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26889,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421732,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281693788,"statfs":{"total":289910292480,"available":288454438912,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1709,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003346"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27408,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367162+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367162+0000","last_peered":"2024-09-11T05:25:41.367162+0000","last_clean":"2024-09-11T05:25:41.367162+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367162+0000","last_undegraded":"2024-09-11T05:25:41.367162+0000","last_fullsized":"2024-09-11T05:25:41.367162+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":24290,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:58.460613+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:25:58.460613+0000","last_peered":"2024-09-11T05:25:58.460613+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:25:58.460613+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26888,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491712,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880128,"statfs":{"total":96636764160,"available":96133251072,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215265,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:04.826 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:05.162 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:05.163 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:05.182 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-11T05:26:03.808971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27359,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1409252,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281706268,"statfs":{"total":289910292480,"available":288467218432,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1433,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1433,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004916"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27409,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367235+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367235+0000","last_peered":"2024-09-11T05:25:41.367235+0000","last_clean":"2024-09-11T05:25:41.367235+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367235+0000","last_undegraded":"2024-09-11T05:25:41.367235+0000","last_fullsized":"2024-09-11T05:25:41.367235+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":25230,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:03.462897+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:26:03.462897+0000","last_peered":"2024-09-11T05:26:03.462897+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:26:03.462897+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-913}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27358,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479232,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892608,"statfs":{"total":96636764160,"available":96146030592,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:05.184 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:05.514 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:05.514 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:05.532 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-11T05:26:03.808971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27359,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1409252,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281706268,"statfs":{"total":289910292480,"available":288467218432,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1433,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1433,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004916"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27409,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367235+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367235+0000","last_peered":"2024-09-11T05:25:41.367235+0000","last_clean":"2024-09-11T05:25:41.367235+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367235+0000","last_undegraded":"2024-09-11T05:25:41.367235+0000","last_fullsized":"2024-09-11T05:25:41.367235+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":25230,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:03.462897+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:26:03.462897+0000","last_peered":"2024-09-11T05:26:03.462897+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:26:03.462897+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-913}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27358,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479232,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892608,"statfs":{"total":96636764160,"available":96146030592,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:05.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:05.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20296, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.367899+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:41.367899+0000', 'last_peered': '2024-09-11T05:25:41.367899+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:41.367899+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:05.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:05.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19865, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.360632+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:41.360632+0000', 'last_peered': '2024-09-11T05:25:41.360632+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:41.360632+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:05.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:05.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20303, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.364747+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:41.364747+0000', 'last_peered': '2024-09-11T05:25:41.364747+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:41.364747+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:05.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:05.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19816, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.366260+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:41.366260+0000', 'last_peered': '2024-09-11T05:25:41.366260+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:41.366260+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:05.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:26:05.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 25230, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:03.462897+0000', 'last_change': '2024-09-11T05:25:39.846459+0000', 'last_active': '2024-09-11T05:26:03.462897+0000', 'last_peered': '2024-09-11T05:26:03.462897+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:26:03.462897+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 913, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2633, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -913}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:05.534 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:05.916 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:05.916 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15154/6 objects degraded (252566.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":843,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1443074048,"bytes_avail":288467218432,"bytes_total":289910292480,"degraded_objects":15154,"degraded_total":6,"degraded_ratio":2525.6666666666665,"recovering_objects_per_sec":73,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":888,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:05.938 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:05.938 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:06.247 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:06.248 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:06.267 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-11T05:26:05.809655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27359,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1409252,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281706268,"statfs":{"total":289910292480,"available":288467218432,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":888,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005125"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27409,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367235+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367235+0000","last_peered":"2024-09-11T05:25:41.367235+0000","last_clean":"2024-09-11T05:25:41.367235+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367235+0000","last_undegraded":"2024-09-11T05:25:41.367235+0000","last_fullsized":"2024-09-11T05:25:41.367235+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22659,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368371+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368371+0000","last_peered":"2024-09-11T05:25:41.368371+0000","last_clean":"2024-09-11T05:25:41.368371+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368371+0000","last_undegraded":"2024-09-11T05:25:41.368371+0000","last_fullsized":"2024-09-11T05:25:41.368371+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":25230,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:03.462897+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:26:03.462897+0000","last_peered":"2024-09-11T05:26:03.462897+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:26:03.462897+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-913}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27358,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479232,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892608,"statfs":{"total":96636764160,"available":96146030592,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:09.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:09.586 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:09.586 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:09.604 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-11T05:26:07.810266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27359,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413412,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281702108,"statfs":{"total":289910292480,"available":288462958592,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":888,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004852"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27409,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367235+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367235+0000","last_peered":"2024-09-11T05:25:41.367235+0000","last_clean":"2024-09-11T05:25:41.367235+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367235+0000","last_undegraded":"2024-09-11T05:25:41.367235+0000","last_fullsized":"2024-09-11T05:25:41.367235+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368501+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368501+0000","last_peered":"2024-09-11T05:25:41.368501+0000","last_clean":"2024-09-11T05:25:41.368501+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368501+0000","last_undegraded":"2024-09-11T05:25:41.368501+0000","last_fullsized":"2024-09-11T05:25:41.368501+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20303,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.364747+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.364747+0000","last_peered":"2024-09-11T05:25:41.364747+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.364747+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":25230,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:03.462897+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:26:03.462897+0000","last_peered":"2024-09-11T05:26:03.462897+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:26:03.462897+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-913}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27358,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:09.605 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:09.912 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:09.928 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-11T05:26:09.810872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27913,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413412,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281702108,"statfs":{"total":289910292480,"available":288462958592,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004885"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27409,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367235+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367235+0000","last_peered":"2024-09-11T05:25:41.367235+0000","last_clean":"2024-09-11T05:25:41.367235+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367235+0000","last_undegraded":"2024-09-11T05:25:41.367235+0000","last_fullsized":"2024-09-11T05:25:41.367235+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368501+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368501+0000","last_peered":"2024-09-11T05:25:41.368501+0000","last_clean":"2024-09-11T05:25:41.368501+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368501+0000","last_undegraded":"2024-09-11T05:25:41.368501+0000","last_fullsized":"2024-09-11T05:25:41.368501+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366623+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.366623+0000","last_peered":"2024-09-11T05:25:41.366623+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.366623+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":26338,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:08.466995+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:26:08.466995+0000","last_peered":"2024-09-11T05:26:08.466995+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:26:08.466995+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27912,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215267,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:09.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:09.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20296, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.367899+0000', 'last_change': '2024-09-11T05:14:14.325188+0000', 'last_active': '2024-09-11T05:25:41.367899+0000', 'last_peered': '2024-09-11T05:25:41.367899+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:25:41.367899+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:09.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:09.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19865, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.360632+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:25:41.360632+0000', 'last_peered': '2024-09-11T05:25:41.360632+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:25:41.360632+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:09.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:09.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20304, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.366623+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:25:41.366623+0000', 'last_peered': '2024-09-11T05:25:41.366623+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:25:41.366623+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:09.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:09.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19816, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:25:41.366260+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:25:41.366260+0000', 'last_peered': '2024-09-11T05:25:41.366260+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:25:41.366260+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:09.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T05:26:09.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19800", 'reported_seq': 26338, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:08.466995+0000', 'last_change': '2024-09-11T05:25:39.846459+0000', 'last_active': '2024-09-11T05:26:08.466995+0000', 'last_peered': '2024-09-11T05:26:08.466995+0000', 'last_clean': '2024-09-11T05:14:12.119953+0000', 'last_became_active': '2024-09-11T05:14:14.286975+0000', 'last_became_peered': '2024-09-11T05:14:14.286975+0000', 'last_unstale': '2024-09-11T05:26:08.466995+0000', 'last_undegraded': '2024-09-11T05:14:13.881398+0000', 'last_fullsized': '2024-09-11T05:14:13.743223+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3288", 'last_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:08.586771+0000', 'objects_scrubbed': 3238, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1359, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 359, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16558, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3187, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:09.930 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:10.297 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:10.298 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14600/6 objects degraded (243333.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":848,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1447333888,"bytes_avail":288462958592,"bytes_total":289910292480,"degraded_objects":14600,"degraded_total":6,"degraded_ratio":2433.3333333333335,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1442,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:10.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:10.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:10.619 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:10.620 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:10.636 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-11T05:26:09.810872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27913,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413412,"kb_used_data":128616,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281702108,"statfs":{"total":289910292480,"available":288462958592,"internally_reserved":0,"allocated":131702784,"data_stored":130722413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004885"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20296,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.367899+0000","last_change":"2024-09-11T05:14:14.325188+0000","last_active":"2024-09-11T05:25:41.367899+0000","last_peered":"2024-09-11T05:25:41.367899+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:25:41.367899+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20030,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374774+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:25:41.374774+0000","last_peered":"2024-09-11T05:25:41.374774+0000","last_clean":"2024-09-11T05:25:41.374774+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:25:41.374774+0000","last_undegraded":"2024-09-11T05:25:41.374774+0000","last_fullsized":"2024-09-11T05:25:41.374774+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26872,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367378+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:25:41.367378+0000","last_peered":"2024-09-11T05:25:41.367378+0000","last_clean":"2024-09-11T05:25:41.367378+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:25:41.367378+0000","last_undegraded":"2024-09-11T05:25:41.367378+0000","last_fullsized":"2024-09-11T05:25:41.367378+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374268+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:25:41.374268+0000","last_peered":"2024-09-11T05:25:41.374268+0000","last_clean":"2024-09-11T05:25:41.374268+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:25:41.374268+0000","last_undegraded":"2024-09-11T05:25:41.374268+0000","last_fullsized":"2024-09-11T05:25:41.374268+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20326,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.364317+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:25:41.364317+0000","last_peered":"2024-09-11T05:25:41.364317+0000","last_clean":"2024-09-11T05:25:41.364317+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:25:41.364317+0000","last_undegraded":"2024-09-11T05:25:41.364317+0000","last_fullsized":"2024-09-11T05:25:41.364317+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27409,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367235+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:25:41.367235+0000","last_peered":"2024-09-11T05:25:41.367235+0000","last_clean":"2024-09-11T05:25:41.367235+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:25:41.367235+0000","last_undegraded":"2024-09-11T05:25:41.367235+0000","last_fullsized":"2024-09-11T05:25:41.367235+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22660,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.368501+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:25:41.368501+0000","last_peered":"2024-09-11T05:25:41.368501+0000","last_clean":"2024-09-11T05:25:41.368501+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:25:41.368501+0000","last_undegraded":"2024-09-11T05:25:41.368501+0000","last_fullsized":"2024-09-11T05:25:41.368501+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20036,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.359708+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:25:41.359708+0000","last_peered":"2024-09-11T05:25:41.359708+0000","last_clean":"2024-09-11T05:25:41.359708+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:25:41.359708+0000","last_undegraded":"2024-09-11T05:25:41.359708+0000","last_fullsized":"2024-09-11T05:25:41.359708+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367036+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:25:41.367036+0000","last_peered":"2024-09-11T05:25:41.367036+0000","last_clean":"2024-09-11T05:25:41.367036+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:25:41.367036+0000","last_undegraded":"2024-09-11T05:25:41.367036+0000","last_fullsized":"2024-09-11T05:25:41.367036+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20124,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.467272+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:25:41.467272+0000","last_peered":"2024-09-11T05:25:41.467272+0000","last_clean":"2024-09-11T05:25:41.467272+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:25:41.467272+0000","last_undegraded":"2024-09-11T05:25:41.467272+0000","last_fullsized":"2024-09-11T05:25:41.467272+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373601+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:25:41.373601+0000","last_peered":"2024-09-11T05:25:41.373601+0000","last_clean":"2024-09-11T05:25:41.373601+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:25:41.373601+0000","last_undegraded":"2024-09-11T05:25:41.373601+0000","last_fullsized":"2024-09-11T05:25:41.373601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19865,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.360632+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:25:41.360632+0000","last_peered":"2024-09-11T05:25:41.360632+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:25:41.360632+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.360216+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:25:41.360216+0000","last_peered":"2024-09-11T05:25:41.360216+0000","last_clean":"2024-09-11T05:25:41.360216+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:25:41.360216+0000","last_undegraded":"2024-09-11T05:25:41.360216+0000","last_fullsized":"2024-09-11T05:25:41.360216+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27235,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.373734+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:25:41.373734+0000","last_peered":"2024-09-11T05:25:41.373734+0000","last_clean":"2024-09-11T05:25:41.373734+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:25:41.373734+0000","last_undegraded":"2024-09-11T05:25:41.373734+0000","last_fullsized":"2024-09-11T05:25:41.373734+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367502+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:25:41.367502+0000","last_peered":"2024-09-11T05:25:41.367502+0000","last_clean":"2024-09-11T05:25:41.367502+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:25:41.367502+0000","last_undegraded":"2024-09-11T05:25:41.367502+0000","last_fullsized":"2024-09-11T05:25:41.367502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26822,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374125+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:25:41.374125+0000","last_peered":"2024-09-11T05:25:41.374125+0000","last_clean":"2024-09-11T05:25:41.374125+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:25:41.374125+0000","last_undegraded":"2024-09-11T05:25:41.374125+0000","last_fullsized":"2024-09-11T05:25:41.374125+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366623+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:25:41.366623+0000","last_peered":"2024-09-11T05:25:41.366623+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:25:41.366623+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367581+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:25:41.367581+0000","last_peered":"2024-09-11T05:25:41.367581+0000","last_clean":"2024-09-11T05:25:41.367581+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:25:41.367581+0000","last_undegraded":"2024-09-11T05:25:41.367581+0000","last_fullsized":"2024-09-11T05:25:41.367581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19816,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:25:41.366260+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:25:41.366260+0000","last_peered":"2024-09-11T05:25:41.366260+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:25:41.366260+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.374674+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:25:41.374674+0000","last_peered":"2024-09-11T05:25:41.374674+0000","last_clean":"2024-09-11T05:25:41.374674+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:25:41.374674+0000","last_undegraded":"2024-09-11T05:25:41.374674+0000","last_fullsized":"2024-09-11T05:25:41.374674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":26338,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:08.466995+0000","last_change":"2024-09-11T05:25:39.846459+0000","last_active":"2024-09-11T05:26:08.466995+0000","last_peered":"2024-09-11T05:26:08.466995+0000","last_clean":"2024-09-11T05:14:12.119953+0000","last_became_active":"2024-09-11T05:14:14.286975+0000","last_became_peered":"2024-09-11T05:14:14.286975+0000","last_unstale":"2024-09-11T05:26:08.466995+0000","last_undegraded":"2024-09-11T05:14:13.881398+0000","last_fullsized":"2024-09-11T05:14:13.743223+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367212+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:25:41.367212+0000","last_peered":"2024-09-11T05:25:41.367212+0000","last_clean":"2024-09-11T05:25:41.367212+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:25:41.367212+0000","last_undegraded":"2024-09-11T05:25:41.367212+0000","last_fullsized":"2024-09-11T05:25:41.367212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367209+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:25:41.367209+0000","last_peered":"2024-09-11T05:25:41.367209+0000","last_clean":"2024-09-11T05:25:41.367209+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:25:41.367209+0000","last_undegraded":"2024-09-11T05:25:41.367209+0000","last_fullsized":"2024-09-11T05:25:41.367209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T05:25:41.367443+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:25:41.367443+0000","last_peered":"2024-09-11T05:25:41.367443+0000","last_clean":"2024-09-11T05:25:41.367443+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:25:41.367443+0000","last_undegraded":"2024-09-11T05:25:41.367443+0000","last_fullsized":"2024-09-11T05:25:41.367443+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27912,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130269184,"data_stored":130241352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418408,"kb_used_data":71572,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953432,"statfs":{"total":96636764160,"available":96208314368,"internally_reserved":0,"allocated":73289728,"data_stored":72974933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483392,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888448,"statfs":{"total":96636764160,"available":96141770752,"internally_reserved":0,"allocated":507904,"data_stored":164679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215267,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511612,"kb_used_data":56548,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860228,"statfs":{"total":96636764160,"available":96112873472,"internally_reserved":0,"allocated":57905152,"data_stored":57582801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72814592,"data_stored":72814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:13.638 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:13.995 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:13.995 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:14.019 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-11T05:26:13.811840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28593,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417620,"kb_used_data":128656,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281697900,"statfs":{"total":289910292480,"available":288458649600,"internally_reserved":0,"allocated":131743744,"data_stored":130736521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002184"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20943,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:13.472579+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:13.472579+0000","last_peered":"2024-09-11T05:26:13.472579+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:13.472579+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914145+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914145+0000","last_peered":"2024-09-11T05:26:11.914145+0000","last_clean":"2024-09-11T05:26:11.914145+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914145+0000","last_undegraded":"2024-09-11T05:26:11.914145+0000","last_fullsized":"2024-09-11T05:26:11.914145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28592,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122408960,"data_stored":122381128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418416,"kb_used_data":71580,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953424,"statfs":{"total":96636764160,"available":96208306176,"internally_reserved":0,"allocated":73297920,"data_stored":72977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487576,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884264,"statfs":{"total":96636764160,"available":96137486336,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64954368,"data_stored":64954368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:14.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:14.374 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:14.375 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:14.393 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-11T05:26:13.811840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28593,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417620,"kb_used_data":128656,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281697900,"statfs":{"total":289910292480,"available":288458649600,"internally_reserved":0,"allocated":131743744,"data_stored":130736521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002184"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20943,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:13.472579+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:13.472579+0000","last_peered":"2024-09-11T05:26:13.472579+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:13.472579+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914145+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914145+0000","last_peered":"2024-09-11T05:26:11.914145+0000","last_clean":"2024-09-11T05:26:11.914145+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914145+0000","last_undegraded":"2024-09-11T05:26:11.914145+0000","last_fullsized":"2024-09-11T05:26:11.914145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28592,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122408960,"data_stored":122381128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418416,"kb_used_data":71580,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953424,"statfs":{"total":96636764160,"available":96208306176,"internally_reserved":0,"allocated":73297920,"data_stored":72977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487576,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884264,"statfs":{"total":96636764160,"available":96137486336,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64954368,"data_stored":64954368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20943, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:13.472579+0000', 'last_change': '2024-09-11T05:26:10.695281+0000', 'last_active': '2024-09-11T05:26:13.472579+0000', 'last_peered': '2024-09-11T05:26:13.472579+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:26:13.472579+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 321, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3354}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19869, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.921341+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:26:11.921341+0000', 'last_peered': '2024-09-11T05:26:11.921341+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:11.921341+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20308, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919022+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:11.919022+0000', 'last_peered': '2024-09-11T05:26:11.919022+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:11.919022+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:14.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19820, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919123+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:11.919123+0000', 'last_peered': '2024-09-11T05:26:11.919123+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:11.919123+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:14.395 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:14.821 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:14.821 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13920/6 objects degraded (232000.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":852,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1451642880,"bytes_avail":288458649600,"bytes_total":289910292480,"degraded_objects":13920,"degraded_total":6,"degraded_ratio":2320,"recovering_objects_per_sec":154,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:14.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:14.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:15.143 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:15.144 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:15.164 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-11T05:26:13.811840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28593,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417620,"kb_used_data":128656,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281697900,"statfs":{"total":289910292480,"available":288458649600,"internally_reserved":0,"allocated":131743744,"data_stored":130736521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002184"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20943,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:13.472579+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:13.472579+0000","last_peered":"2024-09-11T05:26:13.472579+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:13.472579+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914145+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914145+0000","last_peered":"2024-09-11T05:26:11.914145+0000","last_clean":"2024-09-11T05:26:11.914145+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914145+0000","last_undegraded":"2024-09-11T05:26:11.914145+0000","last_fullsized":"2024-09-11T05:26:11.914145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28592,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122408960,"data_stored":122381128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418416,"kb_used_data":71580,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953424,"statfs":{"total":96636764160,"available":96208306176,"internally_reserved":0,"allocated":73297920,"data_stored":72977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487576,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884264,"statfs":{"total":96636764160,"available":96137486336,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64954368,"data_stored":64954368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:18.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:18.503 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:18.525 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-11T05:26:17.812867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28593,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1409292,"kb_used_data":128656,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281706228,"statfs":{"total":289910292480,"available":288467177472,"internally_reserved":0,"allocated":131743744,"data_stored":130736521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001995"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20943,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:13.472579+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:13.472579+0000","last_peered":"2024-09-11T05:26:13.472579+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:13.472579+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28592,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122408960,"data_stored":122381128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418416,"kb_used_data":71580,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953424,"statfs":{"total":96636764160,"available":96208306176,"internally_reserved":0,"allocated":73297920,"data_stored":72977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64954368,"data_stored":64954368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:18.526 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:18.854 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:18.854 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:18.877 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-11T05:26:17.812867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28593,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1409292,"kb_used_data":128656,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281706228,"statfs":{"total":289910292480,"available":288467177472,"internally_reserved":0,"allocated":131743744,"data_stored":130736521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001995"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20943,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:13.472579+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:13.472579+0000","last_peered":"2024-09-11T05:26:13.472579+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:13.472579+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28592,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122408960,"data_stored":122381128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418416,"kb_used_data":71580,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953424,"statfs":{"total":96636764160,"available":96208306176,"internally_reserved":0,"allocated":73297920,"data_stored":72977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64954368,"data_stored":64954368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:18.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:18.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 20943, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:13.472579+0000', 'last_change': '2024-09-11T05:26:10.695281+0000', 'last_active': '2024-09-11T05:26:13.472579+0000', 'last_peered': '2024-09-11T05:26:13.472579+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:26:13.472579+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3354, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 321, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3354}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:18.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:18.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19869, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.921341+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:26:11.921341+0000', 'last_peered': '2024-09-11T05:26:11.921341+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:11.921341+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:18.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:18.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20308, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919022+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:11.919022+0000', 'last_peered': '2024-09-11T05:26:11.919022+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:11.919022+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:18.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:18.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19820, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919123+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:11.919123+0000', 'last_peered': '2024-09-11T05:26:11.919123+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:11.919123+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:18.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:19.278 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:19.278 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13920/6 objects degraded (232000.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":857,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1443115008,"bytes_avail":288467177472,"bytes_total":289910292480,"degraded_objects":13920,"degraded_total":6,"degraded_ratio":2320,"recovering_objects_per_sec":84,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":680,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:19.300 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:19.300 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:19.643 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:19.662 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-11T05:26:17.812867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28593,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1409292,"kb_used_data":128656,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281706228,"statfs":{"total":289910292480,"available":288467177472,"internally_reserved":0,"allocated":131743744,"data_stored":130736521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001995"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":20943,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:13.472579+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:13.472579+0000","last_peered":"2024-09-11T05:26:13.472579+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:13.472579+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28592,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122408960,"data_stored":122381128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418416,"kb_used_data":71580,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93953424,"statfs":{"total":96636764160,"available":96208306176,"internally_reserved":0,"allocated":73297920,"data_stored":72977763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64954368,"data_stored":64954368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:22.665 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:23.004 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:23.005 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:23.023 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-11T05:26:21.813781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29320,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395132,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281720388,"statfs":{"total":289910292480,"available":288481677312,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002499"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":22397,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:18.482519+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:18.482519+0000","last_peered":"2024-09-11T05:26:18.482519+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:18.482519+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1048,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2627}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29319,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:23.024 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:23.339 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:23.358 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-11T05:26:21.813781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29320,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395132,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281720388,"statfs":{"total":289910292480,"available":288481677312,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002499"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":22397,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:18.482519+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:18.482519+0000","last_peered":"2024-09-11T05:26:18.482519+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:18.482519+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1048,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2627}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29319,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:23.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:23.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 22397, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:18.482519+0000', 'last_change': '2024-09-11T05:26:10.695281+0000', 'last_active': '2024-09-11T05:26:18.482519+0000', 'last_peered': '2024-09-11T05:26:18.482519+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:26:18.482519+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2627, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1048, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2627}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:23.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:23.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19869, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.921341+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:26:11.921341+0000', 'last_peered': '2024-09-11T05:26:11.921341+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:11.921341+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20308, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919022+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:11.919022+0000', 'last_peered': '2024-09-11T05:26:11.919022+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:11.919022+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19820, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919123+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:11.919123+0000', 'last_peered': '2024-09-11T05:26:11.919123+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:11.919123+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:23.360 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:23.805 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:23.806 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13193/6 objects degraded (219883.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":861,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1428615168,"bytes_avail":288481677312,"bytes_total":289910292480,"degraded_objects":13193,"degraded_total":6,"degraded_ratio":2198.8333333333335,"recovering_objects_per_sec":140,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1407,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:23.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:23.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:24.166 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:24.184 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-11T05:26:23.814501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30047,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399292,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281716228,"statfs":{"total":289910292480,"available":288477417472,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.905569"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":23851,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:23.481616+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:23.481616+0000","last_peered":"2024-09-11T05:26:23.481616+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:23.481616+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1775,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30046,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215270,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:27.186 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:27.520 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:27.520 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:27.540 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-09-11T05:26:25.815033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30047,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399292,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281716228,"statfs":{"total":289910292480,"available":288477417472,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1454,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003193"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":23851,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:23.481616+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:23.481616+0000","last_peered":"2024-09-11T05:26:23.481616+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:23.481616+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1775,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30046,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483408,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888432,"statfs":{"total":96636764160,"available":96141754368,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215270,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:27.541 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:27.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:27.878 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:27.896 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-11T05:26:27.815582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30047,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403452,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281712068,"statfs":{"total":289910292480,"available":288473157632,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1454,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003287"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":23851,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:23.481616+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:23.481616+0000","last_peered":"2024-09-11T05:26:23.481616+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:23.481616+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1775,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30046,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487568,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884272,"statfs":{"total":96636764160,"available":96137494528,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215270,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:27.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:27.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 23851, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:23.481616+0000', 'last_change': '2024-09-11T05:26:10.695281+0000', 'last_active': '2024-09-11T05:26:23.481616+0000', 'last_peered': '2024-09-11T05:26:23.481616+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:26:23.481616+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1900, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1775, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1900}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:27.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:27.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19869, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.921341+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:26:11.921341+0000', 'last_peered': '2024-09-11T05:26:11.921341+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:11.921341+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:27.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:27.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20308, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919022+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:11.919022+0000', 'last_peered': '2024-09-11T05:26:11.919022+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:11.919022+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:27.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:27.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19820, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919123+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:11.919123+0000', 'last_peered': '2024-09-11T05:26:11.919123+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:11.919123+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:27.899 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:28.285 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:28.286 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12466/6 objects degraded (207766.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":866,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1432875008,"bytes_avail":288477417472,"bytes_total":289910292480,"degraded_objects":12466,"degraded_total":6,"degraded_ratio":2077.6666666666665,"recovering_objects_per_sec":121,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1454,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:28.305 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:28.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:28.646 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:28.668 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-11T05:26:27.815582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30047,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403452,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281712068,"statfs":{"total":289910292480,"available":288473157632,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1454,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1454,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003287"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":23851,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:23.481616+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:23.481616+0000","last_peered":"2024-09-11T05:26:23.481616+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:23.481616+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1775,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1900}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30046,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487568,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884272,"statfs":{"total":96636764160,"available":96137494528,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215270,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:31.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:31.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:31.994 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:32.012 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-11T05:26:31.816916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30459,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403452,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281712068,"statfs":{"total":289910292480,"available":288473157632,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003556"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":24676,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:28.483680+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:28.483680+0000","last_peered":"2024-09-11T05:26:28.483680+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:28.483680+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1488}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30458,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487568,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884272,"statfs":{"total":96636764160,"available":96137494528,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215271,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:32.013 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:32.331 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:32.331 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:32.349 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-11T05:26:31.816916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30459,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403452,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281712068,"statfs":{"total":289910292480,"available":288473157632,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003556"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":24676,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:28.483680+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:28.483680+0000","last_peered":"2024-09-11T05:26:28.483680+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:28.483680+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1488}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30458,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487568,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884272,"statfs":{"total":96636764160,"available":96137494528,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215271,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 24676, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:28.483680+0000', 'last_change': '2024-09-11T05:26:10.695281+0000', 'last_active': '2024-09-11T05:26:28.483680+0000', 'last_peered': '2024-09-11T05:26:28.483680+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:26:28.483680+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2187, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1488}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19869, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.921341+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:26:11.921341+0000', 'last_peered': '2024-09-11T05:26:11.921341+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:11.921341+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20308, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919022+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:11.919022+0000', 'last_peered': '2024-09-11T05:26:11.919022+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:11.919022+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19820, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919123+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:11.919123+0000', 'last_peered': '2024-09-11T05:26:11.919123+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:11.919123+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:32.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:32.725 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:32.725 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12054/6 objects degraded (200900.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":870,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1437134848,"bytes_avail":288473157632,"bytes_total":289910292480,"degraded_objects":12054,"degraded_total":6,"degraded_ratio":2009,"recovering_objects_per_sec":94,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:32.744 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:32.745 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:33.086 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:33.087 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:33.106 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-11T05:26:31.816916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30459,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1403452,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281712068,"statfs":{"total":289910292480,"available":288473157632,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003556"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":24676,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:28.483680+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:28.483680+0000","last_peered":"2024-09-11T05:26:28.483680+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:28.483680+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1488}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30458,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487568,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884272,"statfs":{"total":96636764160,"available":96137494528,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215271,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:36.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:36.435 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:36.436 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:36.454 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-11T05:26:35.818178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30957,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1407612,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281707908,"statfs":{"total":289910292480,"available":288468897792,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003677"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":25672,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:33.480745+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:33.480745+0000","last_peered":"2024-09-11T05:26:33.480745+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:33.480745+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30956,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215272,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:36.455 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:36.777 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:36.777 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:36.799 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-11T05:26:35.818178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30957,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1407612,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281707908,"statfs":{"total":289910292480,"available":288468897792,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003677"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":25672,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:33.480745+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:33.480745+0000","last_peered":"2024-09-11T05:26:33.480745+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:33.480745+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30956,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215272,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:36.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:36.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 25672, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:33.480745+0000', 'last_change': '2024-09-11T05:26:10.695281+0000', 'last_active': '2024-09-11T05:26:33.480745+0000', 'last_peered': '2024-09-11T05:26:33.480745+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:26:33.480745+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 990, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2685, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -990}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:36.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:36.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19869, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.921341+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:26:11.921341+0000', 'last_peered': '2024-09-11T05:26:11.921341+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:11.921341+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:36.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:36.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20308, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919022+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:11.919022+0000', 'last_peered': '2024-09-11T05:26:11.919022+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:11.919022+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:36.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:36.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19820, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919123+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:11.919123+0000', 'last_peered': '2024-09-11T05:26:11.919123+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:11.919123+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:36.802 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:37.259 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:37.260 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11556/6 objects degraded (192600.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":875,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1441394688,"bytes_avail":288468897792,"bytes_total":289910292480,"degraded_objects":11556,"degraded_total":6,"degraded_ratio":1926,"recovering_objects_per_sec":75,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":910,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:37.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:37.279 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:37.579 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:37.580 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:37.597 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-11T05:26:35.818178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30957,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1407612,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281707908,"statfs":{"total":289910292480,"available":288468897792,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003677"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":25672,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:33.480745+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:33.480745+0000","last_peered":"2024-09-11T05:26:33.480745+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:33.480745+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19869,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.921341+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.921341+0000","last_peered":"2024-09-11T05:26:11.921341+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.921341+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30956,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491728,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880112,"statfs":{"total":96636764160,"available":96133234688,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215272,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:40.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:40.930 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:40.949 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-11T05:26:39.819727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31564,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395132,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281720388,"statfs":{"total":289910292480,"available":288481677312,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1517,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004145"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":26886,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:38.486443+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:38.486443+0000","last_peered":"2024-09-11T05:26:38.486443+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:38.486443+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-383}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19870,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.922639+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.922639+0000","last_peered":"2024-09-11T05:26:11.922639+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.922639+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31563,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:40.950 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:41.260 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:41.260 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:41.279 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-11T05:26:39.819727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31564,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395132,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281720388,"statfs":{"total":289910292480,"available":288481677312,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1517,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004145"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":26886,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:38.486443+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:38.486443+0000","last_peered":"2024-09-11T05:26:38.486443+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:38.486443+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-383}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19870,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.922639+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.922639+0000","last_peered":"2024-09-11T05:26:11.922639+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.922639+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31563,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20190", 'reported_seq': 26886, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:38.486443+0000', 'last_change': '2024-09-11T05:26:10.695281+0000', 'last_active': '2024-09-11T05:26:38.486443+0000', 'last_peered': '2024-09-11T05:26:38.486443+0000', 'last_clean': '2024-09-11T05:14:12.143377+0000', 'last_became_active': '2024-09-11T05:14:14.318030+0000', 'last_became_peered': '2024-09-11T05:14:14.318030+0000', 'last_unstale': '2024-09-11T05:26:38.486443+0000', 'last_undegraded': '2024-09-11T05:14:13.914602+0000', 'last_fullsized': '2024-09-11T05:14:13.742929+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 383, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20190, 'num_write_kb': 40380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -383}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 19870, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.922639+0000', 'last_change': '2024-09-11T05:14:14.335611+0000', 'last_active': '2024-09-11T05:26:11.922639+0000', 'last_peered': '2024-09-11T05:26:11.922639+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:11.922639+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20308, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919022+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:11.919022+0000', 'last_peered': '2024-09-11T05:26:11.919022+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:11.919022+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:41.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19820, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:11.919123+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:11.919123+0000', 'last_peered': '2024-09-11T05:26:11.919123+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:11.919123+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:41.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:41.705 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:41.705 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10949/6 objects degraded (182483.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":879,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1428615168,"bytes_avail":288481677312,"bytes_total":289910292480,"degraded_objects":10949,"degraded_total":6,"degraded_ratio":1824.833333333333,"recovering_objects_per_sec":126,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1517,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:41.725 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:41.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:42.095 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:42.096 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:42.117 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-11T05:26:41.820374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31564,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395132,"kb_used_data":114492,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281720388,"statfs":{"total":289910292480,"available":288481677312,"internally_reserved":0,"allocated":117239808,"data_stored":116227202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003457"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":26886,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:38.486443+0000","last_change":"2024-09-11T05:26:10.695281+0000","last_active":"2024-09-11T05:26:38.486443+0000","last_peered":"2024-09-11T05:26:38.486443+0000","last_clean":"2024-09-11T05:14:12.143377+0000","last_became_active":"2024-09-11T05:14:14.318030+0000","last_became_peered":"2024-09-11T05:14:14.318030+0000","last_unstale":"2024-09-11T05:26:38.486443+0000","last_undegraded":"2024-09-11T05:14:13.914602+0000","last_fullsized":"2024-09-11T05:14:13.742929+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-383}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20034,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908609+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:12.908609+0000","last_peered":"2024-09-11T05:26:12.908609+0000","last_clean":"2024-09-11T05:26:12.908609+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:12.908609+0000","last_undegraded":"2024-09-11T05:26:12.908609+0000","last_fullsized":"2024-09-11T05:26:12.908609+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26876,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914385+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:11.914385+0000","last_peered":"2024-09-11T05:26:11.914385+0000","last_clean":"2024-09-11T05:26:11.914385+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:11.914385+0000","last_undegraded":"2024-09-11T05:26:11.914385+0000","last_fullsized":"2024-09-11T05:26:11.914385+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20063,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908713+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:12.908713+0000","last_peered":"2024-09-11T05:26:12.908713+0000","last_clean":"2024-09-11T05:26:12.908713+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:12.908713+0000","last_undegraded":"2024-09-11T05:26:12.908713+0000","last_fullsized":"2024-09-11T05:26:12.908713+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20331,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.919152+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:11.919152+0000","last_peered":"2024-09-11T05:26:11.919152+0000","last_clean":"2024-09-11T05:26:11.919152+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:11.919152+0000","last_undegraded":"2024-09-11T05:26:11.919152+0000","last_fullsized":"2024-09-11T05:26:11.919152+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27413,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914833+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:11.914833+0000","last_peered":"2024-09-11T05:26:11.914833+0000","last_clean":"2024-09-11T05:26:11.914833+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:11.914833+0000","last_undegraded":"2024-09-11T05:26:11.914833+0000","last_fullsized":"2024-09-11T05:26:11.914833+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22664,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914568+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:11.914568+0000","last_peered":"2024-09-11T05:26:11.914568+0000","last_clean":"2024-09-11T05:26:11.914568+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:11.914568+0000","last_undegraded":"2024-09-11T05:26:11.914568+0000","last_fullsized":"2024-09-11T05:26:11.914568+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20040,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918973+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:11.918973+0000","last_peered":"2024-09-11T05:26:11.918973+0000","last_clean":"2024-09-11T05:26:11.918973+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:11.918973+0000","last_undegraded":"2024-09-11T05:26:11.918973+0000","last_fullsized":"2024-09-11T05:26:11.918973+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914221+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:11.914221+0000","last_peered":"2024-09-11T05:26:11.914221+0000","last_clean":"2024-09-11T05:26:11.914221+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:11.914221+0000","last_undegraded":"2024-09-11T05:26:11.914221+0000","last_fullsized":"2024-09-11T05:26:11.914221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20128,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915313+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:11.915313+0000","last_peered":"2024-09-11T05:26:11.915313+0000","last_clean":"2024-09-11T05:26:11.915313+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:11.915313+0000","last_undegraded":"2024-09-11T05:26:11.915313+0000","last_fullsized":"2024-09-11T05:26:11.915313+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908201+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:12.908201+0000","last_peered":"2024-09-11T05:26:12.908201+0000","last_clean":"2024-09-11T05:26:12.908201+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:12.908201+0000","last_undegraded":"2024-09-11T05:26:12.908201+0000","last_fullsized":"2024-09-11T05:26:12.908201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":19870,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.922639+0000","last_change":"2024-09-11T05:14:14.335611+0000","last_active":"2024-09-11T05:26:11.922639+0000","last_peered":"2024-09-11T05:26:11.922639+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:11.922639+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.918763+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:11.918763+0000","last_peered":"2024-09-11T05:26:11.918763+0000","last_clean":"2024-09-11T05:26:11.918763+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:11.918763+0000","last_undegraded":"2024-09-11T05:26:11.918763+0000","last_fullsized":"2024-09-11T05:26:11.918763+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27239,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907862+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:12.907862+0000","last_peered":"2024-09-11T05:26:12.907862+0000","last_clean":"2024-09-11T05:26:12.907862+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:12.907862+0000","last_undegraded":"2024-09-11T05:26:12.907862+0000","last_fullsized":"2024-09-11T05:26:12.907862+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915154+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:11.915154+0000","last_peered":"2024-09-11T05:26:11.915154+0000","last_clean":"2024-09-11T05:26:11.915154+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:11.915154+0000","last_undegraded":"2024-09-11T05:26:11.915154+0000","last_fullsized":"2024-09-11T05:26:11.915154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26826,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.907972+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:12.907972+0000","last_peered":"2024-09-11T05:26:12.907972+0000","last_clean":"2024-09-11T05:26:12.907972+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:12.907972+0000","last_undegraded":"2024-09-11T05:26:12.907972+0000","last_fullsized":"2024-09-11T05:26:12.907972+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919022+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:11.919022+0000","last_peered":"2024-09-11T05:26:11.919022+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:11.919022+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915587+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:11.915587+0000","last_peered":"2024-09-11T05:26:11.915587+0000","last_clean":"2024-09-11T05:26:11.915587+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:11.915587+0000","last_undegraded":"2024-09-11T05:26:11.915587+0000","last_fullsized":"2024-09-11T05:26:11.915587+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19820,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:11.919123+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:11.919123+0000","last_peered":"2024-09-11T05:26:11.919123+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:11.919123+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.908477+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:12.908477+0000","last_peered":"2024-09-11T05:26:12.908477+0000","last_clean":"2024-09-11T05:26:12.908477+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:12.908477+0000","last_undegraded":"2024-09-11T05:26:12.908477+0000","last_fullsized":"2024-09-11T05:26:12.908477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19959,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:12.066470+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:12.066470+0000","last_peered":"2024-09-11T05:26:12.066470+0000","last_clean":"2024-09-11T05:26:12.066470+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:12.066470+0000","last_undegraded":"2024-09-11T05:26:12.066470+0000","last_fullsized":"2024-09-11T05:26:12.066470+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914711+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:11.914711+0000","last_peered":"2024-09-11T05:26:11.914711+0000","last_clean":"2024-09-11T05:26:11.914711+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:11.914711+0000","last_undegraded":"2024-09-11T05:26:11.914711+0000","last_fullsized":"2024-09-11T05:26:11.914711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.914452+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:11.914452+0000","last_peered":"2024-09-11T05:26:11.914452+0000","last_clean":"2024-09-11T05:26:11.914452+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:11.914452+0000","last_undegraded":"2024-09-11T05:26:11.914452+0000","last_fullsized":"2024-09-11T05:26:11.914452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19763,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T05:26:11.915462+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:11.915462+0000","last_peered":"2024-09-11T05:26:11.915462+0000","last_clean":"2024-09-11T05:26:11.915462+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:11.915462+0000","last_undegraded":"2024-09-11T05:26:11.915462+0000","last_fullsized":"2024-09-11T05:26:11.915462+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31563,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115757056,"data_stored":115729224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404256,"kb_used_data":57416,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93967584,"statfs":{"total":96636764160,"available":96222806016,"internally_reserved":0,"allocated":58793984,"data_stored":58468444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479248,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892592,"statfs":{"total":96636764160,"available":96146014208,"internally_reserved":0,"allocated":524288,"data_stored":170318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511628,"kb_used_data":56564,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860212,"statfs":{"total":96636764160,"available":96112857088,"internally_reserved":0,"allocated":57921536,"data_stored":57588440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58302464,"data_stored":58302464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:45.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:45.502 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:45.502 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-11T05:26:43.820670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32226,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1394852,"kb_used_data":105768,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281720668,"statfs":{"total":289910292480,"available":288481964032,"internally_reserved":0,"allocated":108306432,"data_stored":107273807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002491"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20038,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887636+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887636+0000","last_peered":"2024-09-11T05:26:42.887636+0000","last_clean":"2024-09-11T05:26:42.887636+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887636+0000","last_undegraded":"2024-09-11T05:26:42.887636+0000","last_fullsized":"2024-09-11T05:26:42.887636+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522198+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:41.522198+0000","last_peered":"2024-09-11T05:26:41.522198+0000","last_clean":"2024-09-11T05:26:41.522198+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:41.522198+0000","last_undegraded":"2024-09-11T05:26:41.522198+0000","last_fullsized":"2024-09-11T05:26:41.522198+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20067,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887612+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887612+0000","last_peered":"2024-09-11T05:26:42.887612+0000","last_clean":"2024-09-11T05:26:42.887612+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887612+0000","last_undegraded":"2024-09-11T05:26:42.887612+0000","last_fullsized":"2024-09-11T05:26:42.887612+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27415,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.523163+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:41.523163+0000","last_peered":"2024-09-11T05:26:41.523163+0000","last_clean":"2024-09-11T05:26:41.523163+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:41.523163+0000","last_undegraded":"2024-09-11T05:26:41.523163+0000","last_fullsized":"2024-09-11T05:26:41.523163+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22666,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522989+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:41.522989+0000","last_peered":"2024-09-11T05:26:41.522989+0000","last_clean":"2024-09-11T05:26:41.522989+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:41.522989+0000","last_undegraded":"2024-09-11T05:26:41.522989+0000","last_fullsized":"2024-09-11T05:26:41.522989+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522104+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:41.522104+0000","last_peered":"2024-09-11T05:26:41.522104+0000","last_clean":"2024-09-11T05:26:41.522104+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:41.522104+0000","last_undegraded":"2024-09-11T05:26:41.522104+0000","last_fullsized":"2024-09-11T05:26:41.522104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522225+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:41.522225+0000","last_peered":"2024-09-11T05:26:41.522225+0000","last_clean":"2024-09-11T05:26:41.522225+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:41.522225+0000","last_undegraded":"2024-09-11T05:26:41.522225+0000","last_fullsized":"2024-09-11T05:26:41.522225+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":20433,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:43.487118+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:43.487118+0000","last_peered":"2024-09-11T05:26:43.487118+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:43.487118+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522105+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:41.522105+0000","last_peered":"2024-09-11T05:26:41.522105+0000","last_clean":"2024-09-11T05:26:41.522105+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:41.522105+0000","last_undegraded":"2024-09-11T05:26:41.522105+0000","last_fullsized":"2024-09-11T05:26:41.522105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522919+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:41.522919+0000","last_peered":"2024-09-11T05:26:41.522919+0000","last_clean":"2024-09-11T05:26:41.522919+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:41.522919+0000","last_undegraded":"2024-09-11T05:26:41.522919+0000","last_fullsized":"2024-09-11T05:26:41.522919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19961,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522420+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:41.522420+0000","last_peered":"2024-09-11T05:26:41.522420+0000","last_clean":"2024-09-11T05:26:41.522420+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:41.522420+0000","last_undegraded":"2024-09-11T05:26:41.522420+0000","last_fullsized":"2024-09-11T05:26:41.522420+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.524151+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:41.524151+0000","last_peered":"2024-09-11T05:26:41.524151+0000","last_clean":"2024-09-11T05:26:41.524151+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:41.524151+0000","last_undegraded":"2024-09-11T05:26:41.524151+0000","last_fullsized":"2024-09-11T05:26:41.524151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522251+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:41.522251+0000","last_peered":"2024-09-11T05:26:41.522251+0000","last_clean":"2024-09-11T05:26:41.522251+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:41.522251+0000","last_undegraded":"2024-09-11T05:26:41.522251+0000","last_fullsized":"2024-09-11T05:26:41.522251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19765,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522207+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:41.522207+0000","last_peered":"2024-09-11T05:26:41.522207+0000","last_clean":"2024-09-11T05:26:41.522207+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:41.522207+0000","last_undegraded":"2024-09-11T05:26:41.522207+0000","last_fullsized":"2024-09-11T05:26:41.522207+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32225,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":105185280,"data_stored":105164406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395636,"kb_used_data":48676,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93976204,"statfs":{"total":96636764160,"available":96231632896,"internally_reserved":0,"allocated":49844224,"data_stored":49507889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":524288,"data_stored":171889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47738880,"data_stored":47738880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:45.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:45.847 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:45.848 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:45.869 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-11T05:26:45.821013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32226,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1394852,"kb_used_data":105768,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281720668,"statfs":{"total":289910292480,"available":288481964032,"internally_reserved":0,"allocated":108306432,"data_stored":107273807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002137"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20038,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887636+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887636+0000","last_peered":"2024-09-11T05:26:42.887636+0000","last_clean":"2024-09-11T05:26:42.887636+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887636+0000","last_undegraded":"2024-09-11T05:26:42.887636+0000","last_fullsized":"2024-09-11T05:26:42.887636+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522198+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:41.522198+0000","last_peered":"2024-09-11T05:26:41.522198+0000","last_clean":"2024-09-11T05:26:41.522198+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:41.522198+0000","last_undegraded":"2024-09-11T05:26:41.522198+0000","last_fullsized":"2024-09-11T05:26:41.522198+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20067,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887612+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887612+0000","last_peered":"2024-09-11T05:26:42.887612+0000","last_clean":"2024-09-11T05:26:42.887612+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887612+0000","last_undegraded":"2024-09-11T05:26:42.887612+0000","last_fullsized":"2024-09-11T05:26:42.887612+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27415,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.523163+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:41.523163+0000","last_peered":"2024-09-11T05:26:41.523163+0000","last_clean":"2024-09-11T05:26:41.523163+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:41.523163+0000","last_undegraded":"2024-09-11T05:26:41.523163+0000","last_fullsized":"2024-09-11T05:26:41.523163+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22666,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522989+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:41.522989+0000","last_peered":"2024-09-11T05:26:41.522989+0000","last_clean":"2024-09-11T05:26:41.522989+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:41.522989+0000","last_undegraded":"2024-09-11T05:26:41.522989+0000","last_fullsized":"2024-09-11T05:26:41.522989+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522104+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:41.522104+0000","last_peered":"2024-09-11T05:26:41.522104+0000","last_clean":"2024-09-11T05:26:41.522104+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:41.522104+0000","last_undegraded":"2024-09-11T05:26:41.522104+0000","last_fullsized":"2024-09-11T05:26:41.522104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522225+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:41.522225+0000","last_peered":"2024-09-11T05:26:41.522225+0000","last_clean":"2024-09-11T05:26:41.522225+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:41.522225+0000","last_undegraded":"2024-09-11T05:26:41.522225+0000","last_fullsized":"2024-09-11T05:26:41.522225+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":20433,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:43.487118+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:43.487118+0000","last_peered":"2024-09-11T05:26:43.487118+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:43.487118+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522105+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:41.522105+0000","last_peered":"2024-09-11T05:26:41.522105+0000","last_clean":"2024-09-11T05:26:41.522105+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:41.522105+0000","last_undegraded":"2024-09-11T05:26:41.522105+0000","last_fullsized":"2024-09-11T05:26:41.522105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522919+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:41.522919+0000","last_peered":"2024-09-11T05:26:41.522919+0000","last_clean":"2024-09-11T05:26:41.522919+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:41.522919+0000","last_undegraded":"2024-09-11T05:26:41.522919+0000","last_fullsized":"2024-09-11T05:26:41.522919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19961,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522420+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:41.522420+0000","last_peered":"2024-09-11T05:26:41.522420+0000","last_clean":"2024-09-11T05:26:41.522420+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:41.522420+0000","last_undegraded":"2024-09-11T05:26:41.522420+0000","last_fullsized":"2024-09-11T05:26:41.522420+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.524151+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:41.524151+0000","last_peered":"2024-09-11T05:26:41.524151+0000","last_clean":"2024-09-11T05:26:41.524151+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:41.524151+0000","last_undegraded":"2024-09-11T05:26:41.524151+0000","last_fullsized":"2024-09-11T05:26:41.524151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522251+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:41.522251+0000","last_peered":"2024-09-11T05:26:41.522251+0000","last_clean":"2024-09-11T05:26:41.522251+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:41.522251+0000","last_undegraded":"2024-09-11T05:26:41.522251+0000","last_fullsized":"2024-09-11T05:26:41.522251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19765,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522207+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:41.522207+0000","last_peered":"2024-09-11T05:26:41.522207+0000","last_clean":"2024-09-11T05:26:41.522207+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:41.522207+0000","last_undegraded":"2024-09-11T05:26:41.522207+0000","last_fullsized":"2024-09-11T05:26:41.522207+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32225,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":105185280,"data_stored":105164406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395636,"kb_used_data":48676,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93976204,"statfs":{"total":96636764160,"available":96231632896,"internally_reserved":0,"allocated":49844224,"data_stored":49507889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":524288,"data_stored":171889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47738880,"data_stored":47738880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:45.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:45.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 20433, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:43.487118+0000', 'last_change': '2024-09-11T05:26:41.466628+0000', 'last_active': '2024-09-11T05:26:43.487118+0000', 'last_peered': '2024-09-11T05:26:43.487118+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:43.487118+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 279, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3165}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:45.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:45.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20312, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530394+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:42.530394+0000', 'last_peered': '2024-09-11T05:26:42.530394+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:42.530394+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:45.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:45.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19824, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530445+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:42.530445+0000', 'last_peered': '2024-09-11T05:26:42.530445+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:42.530445+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:45.871 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:46.294 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:46.294 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10287/6 objects degraded (171450.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":884,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1428328448,"bytes_avail":288481964032,"bytes_total":289910292480,"degraded_objects":10287,"degraded_total":6,"degraded_ratio":1714.5,"recovering_objects_per_sec":158,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:46.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:46.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:46.685 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-11T05:26:45.821013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32226,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1394852,"kb_used_data":105768,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281720668,"statfs":{"total":289910292480,"available":288481964032,"internally_reserved":0,"allocated":108306432,"data_stored":107273807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002137"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20038,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887636+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887636+0000","last_peered":"2024-09-11T05:26:42.887636+0000","last_clean":"2024-09-11T05:26:42.887636+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887636+0000","last_undegraded":"2024-09-11T05:26:42.887636+0000","last_fullsized":"2024-09-11T05:26:42.887636+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522198+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:41.522198+0000","last_peered":"2024-09-11T05:26:41.522198+0000","last_clean":"2024-09-11T05:26:41.522198+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:41.522198+0000","last_undegraded":"2024-09-11T05:26:41.522198+0000","last_fullsized":"2024-09-11T05:26:41.522198+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20067,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887612+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887612+0000","last_peered":"2024-09-11T05:26:42.887612+0000","last_clean":"2024-09-11T05:26:42.887612+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887612+0000","last_undegraded":"2024-09-11T05:26:42.887612+0000","last_fullsized":"2024-09-11T05:26:42.887612+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27415,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.523163+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:41.523163+0000","last_peered":"2024-09-11T05:26:41.523163+0000","last_clean":"2024-09-11T05:26:41.523163+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:41.523163+0000","last_undegraded":"2024-09-11T05:26:41.523163+0000","last_fullsized":"2024-09-11T05:26:41.523163+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22666,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522989+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:41.522989+0000","last_peered":"2024-09-11T05:26:41.522989+0000","last_clean":"2024-09-11T05:26:41.522989+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:41.522989+0000","last_undegraded":"2024-09-11T05:26:41.522989+0000","last_fullsized":"2024-09-11T05:26:41.522989+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522104+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:41.522104+0000","last_peered":"2024-09-11T05:26:41.522104+0000","last_clean":"2024-09-11T05:26:41.522104+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:41.522104+0000","last_undegraded":"2024-09-11T05:26:41.522104+0000","last_fullsized":"2024-09-11T05:26:41.522104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20130,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522225+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:41.522225+0000","last_peered":"2024-09-11T05:26:41.522225+0000","last_clean":"2024-09-11T05:26:41.522225+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:41.522225+0000","last_undegraded":"2024-09-11T05:26:41.522225+0000","last_fullsized":"2024-09-11T05:26:41.522225+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":20433,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:43.487118+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:43.487118+0000","last_peered":"2024-09-11T05:26:43.487118+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:43.487118+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":113,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522105+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:41.522105+0000","last_peered":"2024-09-11T05:26:41.522105+0000","last_clean":"2024-09-11T05:26:41.522105+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:41.522105+0000","last_undegraded":"2024-09-11T05:26:41.522105+0000","last_fullsized":"2024-09-11T05:26:41.522105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522919+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:41.522919+0000","last_peered":"2024-09-11T05:26:41.522919+0000","last_clean":"2024-09-11T05:26:41.522919+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:41.522919+0000","last_undegraded":"2024-09-11T05:26:41.522919+0000","last_fullsized":"2024-09-11T05:26:41.522919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19961,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522420+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:41.522420+0000","last_peered":"2024-09-11T05:26:41.522420+0000","last_clean":"2024-09-11T05:26:41.522420+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:41.522420+0000","last_undegraded":"2024-09-11T05:26:41.522420+0000","last_fullsized":"2024-09-11T05:26:41.522420+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.524151+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:41.524151+0000","last_peered":"2024-09-11T05:26:41.524151+0000","last_clean":"2024-09-11T05:26:41.524151+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:41.524151+0000","last_undegraded":"2024-09-11T05:26:41.524151+0000","last_fullsized":"2024-09-11T05:26:41.524151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522251+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:41.522251+0000","last_peered":"2024-09-11T05:26:41.522251+0000","last_clean":"2024-09-11T05:26:41.522251+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:41.522251+0000","last_undegraded":"2024-09-11T05:26:41.522251+0000","last_fullsized":"2024-09-11T05:26:41.522251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19765,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T05:26:41.522207+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:41.522207+0000","last_peered":"2024-09-11T05:26:41.522207+0000","last_clean":"2024-09-11T05:26:41.522207+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:41.522207+0000","last_undegraded":"2024-09-11T05:26:41.522207+0000","last_fullsized":"2024-09-11T05:26:41.522207+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32225,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":105185280,"data_stored":105164406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395636,"kb_used_data":48676,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93976204,"statfs":{"total":96636764160,"available":96231632896,"internally_reserved":0,"allocated":49844224,"data_stored":49507889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":524288,"data_stored":171889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":47738880,"data_stored":47738880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:49.687 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:50.059 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:50.060 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:50.079 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-11T05:26:49.821981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32967,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1392956,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281722564,"statfs":{"total":289910292480,"available":288483905536,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1403,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.302492"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26880,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527410+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527410+0000","last_peered":"2024-09-11T05:26:42.527410+0000","last_clean":"2024-09-11T05:26:42.527410+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527410+0000","last_undegraded":"2024-09-11T05:26:42.527410+0000","last_fullsized":"2024-09-11T05:26:42.527410+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":21915,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:48.485254+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:48.485254+0000","last_peered":"2024-09-11T05:26:48.485254+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:48.485254+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2424,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1020,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32966,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215275,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:50.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:50.428 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:50.428 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:50.447 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-11T05:26:49.821981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32967,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1392956,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281722564,"statfs":{"total":289910292480,"available":288483905536,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1403,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.302492"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26880,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527410+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527410+0000","last_peered":"2024-09-11T05:26:42.527410+0000","last_clean":"2024-09-11T05:26:42.527410+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527410+0000","last_undegraded":"2024-09-11T05:26:42.527410+0000","last_fullsized":"2024-09-11T05:26:42.527410+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":21915,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:48.485254+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:48.485254+0000","last_peered":"2024-09-11T05:26:48.485254+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:48.485254+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2424,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1020,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32966,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215275,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:50.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:50.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 21915, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:48.485254+0000', 'last_change': '2024-09-11T05:26:41.466628+0000', 'last_active': '2024-09-11T05:26:48.485254+0000', 'last_peered': '2024-09-11T05:26:48.485254+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:48.485254+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2424, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1020, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2424}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:50.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:50.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20312, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530394+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:42.530394+0000', 'last_peered': '2024-09-11T05:26:42.530394+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:42.530394+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:50.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:50.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19824, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530445+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:42.530445+0000', 'last_peered': '2024-09-11T05:26:42.530445+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:42.530445+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:50.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:50.860 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:50.860 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9546/6 objects degraded (159100.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":888,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1426386944,"bytes_avail":288483905536,"bytes_total":289910292480,"degraded_objects":9546,"degraded_total":6,"degraded_ratio":1591,"recovering_objects_per_sec":168,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1403,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:50.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:50.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:51.181 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:51.198 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-11T05:26:49.821981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32967,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1392956,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281722564,"statfs":{"total":289910292480,"available":288483905536,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1403,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.302492"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26880,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527410+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527410+0000","last_peered":"2024-09-11T05:26:42.527410+0000","last_clean":"2024-09-11T05:26:42.527410+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527410+0000","last_undegraded":"2024-09-11T05:26:42.527410+0000","last_fullsized":"2024-09-11T05:26:42.527410+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":21915,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:48.485254+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:48.485254+0000","last_peered":"2024-09-11T05:26:48.485254+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:48.485254+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2424,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1020,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32966,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215275,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:54.200 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:54.519 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:54.537 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-11T05:26:53.823165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33527,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1380476,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281735044,"statfs":{"total":289910292480,"available":288496685056,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.298892"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23035,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:53.485335+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:53.485335+0000","last_peered":"2024-09-11T05:26:53.485335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:53.485335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33526,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479256,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892584,"statfs":{"total":96636764160,"available":96146006016,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215276,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:54.539 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:54.849 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:54.867 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-11T05:26:53.823165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33527,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1380476,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281735044,"statfs":{"total":289910292480,"available":288496685056,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.298892"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23035,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:53.485335+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:53.485335+0000","last_peered":"2024-09-11T05:26:53.485335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:53.485335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33526,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479256,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892584,"statfs":{"total":96636764160,"available":96146006016,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215276,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 23035, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:53.485335+0000', 'last_change': '2024-09-11T05:26:41.466628+0000', 'last_active': '2024-09-11T05:26:53.485335+0000', 'last_peered': '2024-09-11T05:26:53.485335+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:53.485335+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1864, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1580, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1864}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20312, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530394+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:42.530394+0000', 'last_peered': '2024-09-11T05:26:42.530394+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:42.530394+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19824, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530445+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:42.530445+0000', 'last_peered': '2024-09-11T05:26:42.530445+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:42.530445+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:54.869 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:55.254 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8986/6 objects degraded (149766.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":893,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1413607424,"bytes_avail":288496685056,"bytes_total":289910292480,"degraded_objects":8986,"degraded_total":6,"degraded_ratio":1497.666666666667,"recovering_objects_per_sec":173,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:55.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:55.274 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:55.623 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:55.624 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-11T05:26:53.823165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33527,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1380476,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281735044,"statfs":{"total":289910292480,"available":288496685056,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.298892"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23035,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:53.485335+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:53.485335+0000","last_peered":"2024-09-11T05:26:53.485335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:53.485335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33526,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479256,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892584,"statfs":{"total":96636764160,"available":96146006016,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215276,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:58.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:58.975 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:58.993 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-11T05:26:57.824618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33527,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384636,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281730884,"statfs":{"total":289910292480,"available":288492425216,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003604"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23035,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:53.485335+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:53.485335+0000","last_peered":"2024-09-11T05:26:53.485335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:53.485335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33526,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215276,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:58.994 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:26:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:59.311 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:26:59.331 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-11T05:26:57.824618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33527,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384636,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281730884,"statfs":{"total":289910292480,"available":288492425216,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003604"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23035,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:53.485335+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:53.485335+0000","last_peered":"2024-09-11T05:26:53.485335+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:53.485335+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33526,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215276,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:26:59.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:26:59.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 23035, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:53.485335+0000', 'last_change': '2024-09-11T05:26:41.466628+0000', 'last_active': '2024-09-11T05:26:53.485335+0000', 'last_peered': '2024-09-11T05:26:53.485335+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:53.485335+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1864, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1580, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1864}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:59.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:26:59.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20312, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530394+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:42.530394+0000', 'last_peered': '2024-09-11T05:26:42.530394+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:42.530394+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:59.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:26:59.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19824, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530445+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:42.530445+0000', 'last_peered': '2024-09-11T05:26:42.530445+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:42.530445+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:26:59.333 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:26:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:26:59.719 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8986/6 objects degraded (149766.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":897,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1417867264,"bytes_avail":288492425216,"bytes_total":289910292480,"degraded_objects":8986,"degraded_total":6,"degraded_ratio":1497.666666666667,"recovering_objects_per_sec":108,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1301,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:26:59.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:26:59.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:00.062 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:00.080 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-11T05:26:59.825427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33933,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384636,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281730884,"statfs":{"total":289910292480,"available":288492425216,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1707,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004019"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23847,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:58.487975+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:58.487975+0000","last_peered":"2024-09-11T05:26:58.487975+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:58.487975+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33932,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215277,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:03.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:03.400 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:03.400 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:03.418 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-11T05:27:01.825847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33933,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384636,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281730884,"statfs":{"total":289910292480,"available":288492425216,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-966,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":966,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003865"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23847,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:58.487975+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:58.487975+0000","last_peered":"2024-09-11T05:26:58.487975+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:58.487975+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33932,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215277,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:03.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:03.742 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:03.742 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:03.761 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-11T05:27:01.825847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33933,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384636,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281730884,"statfs":{"total":289910292480,"available":288492425216,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-966,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":966,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003865"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20044,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529801+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529801+0000","last_peered":"2024-09-11T05:26:42.529801+0000","last_clean":"2024-09-11T05:26:42.529801+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529801+0000","last_undegraded":"2024-09-11T05:26:42.529801+0000","last_fullsized":"2024-09-11T05:26:42.529801+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":23847,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:58.487975+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:26:58.487975+0000","last_peered":"2024-09-11T05:26:58.487975+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:26:58.487975+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1458}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33932,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483416,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888424,"statfs":{"total":96636764160,"available":96141746176,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215277,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:03.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:27:03.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 23847, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:58.487975+0000', 'last_change': '2024-09-11T05:26:41.466628+0000', 'last_active': '2024-09-11T05:26:58.487975+0000', 'last_peered': '2024-09-11T05:26:58.487975+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:26:58.487975+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1986, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1458}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:03.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:03.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20312, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530394+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:42.530394+0000', 'last_peered': '2024-09-11T05:26:42.530394+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:42.530394+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:03.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:03.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19824, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530445+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:42.530445+0000', 'last_peered': '2024-09-11T05:26:42.530445+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:42.530445+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:03.763 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:04.148 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:04.148 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8085/6 objects degraded (134750.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":901,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1422127104,"bytes_avail":288488165376,"bytes_total":289910292480,"degraded_objects":8085,"degraded_total":6,"degraded_ratio":1347.5,"recovering_objects_per_sec":121,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:04.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:04.471 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:04.472 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:04.490 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-11T05:27:03.826390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34428,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1388796,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281726724,"statfs":{"total":289910292480,"available":288488165376,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003969"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20045,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529850+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529850+0000","last_peered":"2024-09-11T05:26:42.529850+0000","last_clean":"2024-09-11T05:26:42.529850+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529850+0000","last_undegraded":"2024-09-11T05:26:42.529850+0000","last_fullsized":"2024-09-11T05:26:42.529850+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":24837,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:03.488524+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:27:03.488524+0000","last_peered":"2024-09-11T05:27:03.488524+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:27:03.488524+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2481,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34427,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487576,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884264,"statfs":{"total":96636764160,"available":96137486336,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:07.492 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:07.816 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:07.816 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:07.834 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-09-11T05:27:05.826770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34428,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1388796,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281726724,"statfs":{"total":289910292480,"available":288488165376,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1319816193},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003604"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20045,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529850+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529850+0000","last_peered":"2024-09-11T05:26:42.529850+0000","last_clean":"2024-09-11T05:26:42.529850+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529850+0000","last_undegraded":"2024-09-11T05:26:42.529850+0000","last_fullsized":"2024-09-11T05:26:42.529850+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":24837,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:03.488524+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:27:03.488524+0000","last_peered":"2024-09-11T05:27:03.488524+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:27:03.488524+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2481,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34427,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487576,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884264,"statfs":{"total":96636764160,"available":96137486336,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":498724523},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:07.835 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:08.151 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:08.151 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:08.170 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-11T05:27:07.827303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34428,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1392956,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281722564,"statfs":{"total":289910292480,"available":288483905536,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003393"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20045,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529850+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529850+0000","last_peered":"2024-09-11T05:26:42.529850+0000","last_clean":"2024-09-11T05:26:42.529850+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529850+0000","last_undegraded":"2024-09-11T05:26:42.529850+0000","last_fullsized":"2024-09-11T05:26:42.529850+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":24837,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:03.488524+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:27:03.488524+0000","last_peered":"2024-09-11T05:27:03.488524+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:27:03.488524+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2481,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34427,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:27:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 24837, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:03.488524+0000', 'last_change': '2024-09-11T05:26:41.466628+0000', 'last_active': '2024-09-11T05:27:03.488524+0000', 'last_peered': '2024-09-11T05:27:03.488524+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:27:03.488524+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 963, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2481, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -963}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20312, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530394+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:42.530394+0000', 'last_peered': '2024-09-11T05:26:42.530394+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:42.530394+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19824, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530445+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:42.530445+0000', 'last_peered': '2024-09-11T05:26:42.530445+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:42.530445+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:08.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:08.551 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:08.552 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8085/6 objects degraded (134750.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":906,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1426386944,"bytes_avail":288483905536,"bytes_total":289910292480,"degraded_objects":8085,"degraded_total":6,"degraded_ratio":1347.5,"recovering_objects_per_sec":75,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:08.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:08.572 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:08.892 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:08.892 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:08.910 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-11T05:27:07.827303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34428,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1392956,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281722564,"statfs":{"total":289910292480,"available":288483905536,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003393"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20045,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529850+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529850+0000","last_peered":"2024-09-11T05:26:42.529850+0000","last_clean":"2024-09-11T05:26:42.529850+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529850+0000","last_undegraded":"2024-09-11T05:26:42.529850+0000","last_fullsized":"2024-09-11T05:26:42.529850+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":24837,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:03.488524+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:27:03.488524+0000","last_peered":"2024-09-11T05:27:03.488524+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:27:03.488524+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2481,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34427,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:11.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:12.338 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:12.359 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-11T05:27:11.828363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35017,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1392956,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281722564,"statfs":{"total":289910292480,"available":288483905536,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1084,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002516"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26881,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527565+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:26:42.527565+0000","last_peered":"2024-09-11T05:26:42.527565+0000","last_clean":"2024-09-11T05:26:42.527565+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:26:42.527565+0000","last_undegraded":"2024-09-11T05:26:42.527565+0000","last_fullsized":"2024-09-11T05:26:42.527565+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27417,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527148+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:26:42.527148+0000","last_peered":"2024-09-11T05:26:42.527148+0000","last_clean":"2024-09-11T05:26:42.527148+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:26:42.527148+0000","last_undegraded":"2024-09-11T05:26:42.527148+0000","last_fullsized":"2024-09-11T05:26:42.527148+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22668,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527011+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:26:42.527011+0000","last_peered":"2024-09-11T05:26:42.527011+0000","last_clean":"2024-09-11T05:26:42.527011+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:26:42.527011+0000","last_undegraded":"2024-09-11T05:26:42.527011+0000","last_fullsized":"2024-09-11T05:26:42.527011+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20045,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529850+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529850+0000","last_peered":"2024-09-11T05:26:42.529850+0000","last_clean":"2024-09-11T05:26:42.529850+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529850+0000","last_undegraded":"2024-09-11T05:26:42.529850+0000","last_fullsized":"2024-09-11T05:26:42.529850+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.526873+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:26:42.526873+0000","last_peered":"2024-09-11T05:26:42.526873+0000","last_clean":"2024-09-11T05:26:42.526873+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:26:42.526873+0000","last_undegraded":"2024-09-11T05:26:42.526873+0000","last_fullsized":"2024-09-11T05:26:42.526873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20132,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527307+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:26:42.527307+0000","last_peered":"2024-09-11T05:26:42.527307+0000","last_clean":"2024-09-11T05:26:42.527307+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:26:42.527307+0000","last_undegraded":"2024-09-11T05:26:42.527307+0000","last_fullsized":"2024-09-11T05:26:42.527307+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26015,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:08.488957+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:27:08.488957+0000","last_peered":"2024-09-11T05:27:08.488957+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:27:08.488957+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-374}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527249+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:26:42.527249+0000","last_peered":"2024-09-11T05:26:42.527249+0000","last_clean":"2024-09-11T05:26:42.527249+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:26:42.527249+0000","last_undegraded":"2024-09-11T05:26:42.527249+0000","last_fullsized":"2024-09-11T05:26:42.527249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527378+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:26:42.527378+0000","last_peered":"2024-09-11T05:26:42.527378+0000","last_clean":"2024-09-11T05:26:42.527378+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:26:42.527378+0000","last_undegraded":"2024-09-11T05:26:42.527378+0000","last_fullsized":"2024-09-11T05:26:42.527378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19963,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527463+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:26:42.527463+0000","last_peered":"2024-09-11T05:26:42.527463+0000","last_clean":"2024-09-11T05:26:42.527463+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:26:42.527463+0000","last_undegraded":"2024-09-11T05:26:42.527463+0000","last_fullsized":"2024-09-11T05:26:42.527463+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527472+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:26:42.527472+0000","last_peered":"2024-09-11T05:26:42.527472+0000","last_clean":"2024-09-11T05:26:42.527472+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:26:42.527472+0000","last_undegraded":"2024-09-11T05:26:42.527472+0000","last_fullsized":"2024-09-11T05:26:42.527472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527409+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:26:42.527409+0000","last_peered":"2024-09-11T05:26:42.527409+0000","last_clean":"2024-09-11T05:26:42.527409+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:26:42.527409+0000","last_undegraded":"2024-09-11T05:26:42.527409+0000","last_fullsized":"2024-09-11T05:26:42.527409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19767,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.527203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:26:42.527203+0000","last_peered":"2024-09-11T05:26:42.527203+0000","last_clean":"2024-09-11T05:26:42.527203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:26:42.527203+0000","last_undegraded":"2024-09-11T05:26:42.527203+0000","last_fullsized":"2024-09-11T05:26:42.527203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35016,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100679286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215279,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:12.361 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:12.725 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:12.749 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-11T05:27:12.543278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35017,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1380476,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281735044,"statfs":{"total":289910292480,"available":288496685056,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-589,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.716887"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26883,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558830+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:11.558830+0000","last_peered":"2024-09-11T05:27:11.558830+0000","last_clean":"2024-09-11T05:27:11.558830+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:11.558830+0000","last_undegraded":"2024-09-11T05:27:11.558830+0000","last_fullsized":"2024-09-11T05:27:11.558830+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27419,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558483+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:11.558483+0000","last_peered":"2024-09-11T05:27:11.558483+0000","last_clean":"2024-09-11T05:27:11.558483+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:11.558483+0000","last_undegraded":"2024-09-11T05:27:11.558483+0000","last_fullsized":"2024-09-11T05:27:11.558483+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22670,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557999+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:11.557999+0000","last_peered":"2024-09-11T05:27:11.557999+0000","last_clean":"2024-09-11T05:27:11.557999+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:11.557999+0000","last_undegraded":"2024-09-11T05:27:11.557999+0000","last_fullsized":"2024-09-11T05:27:11.557999+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20045,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529850+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529850+0000","last_peered":"2024-09-11T05:26:42.529850+0000","last_clean":"2024-09-11T05:26:42.529850+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529850+0000","last_undegraded":"2024-09-11T05:26:42.529850+0000","last_fullsized":"2024-09-11T05:26:42.529850+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557406+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:11.557406+0000","last_peered":"2024-09-11T05:27:11.557406+0000","last_clean":"2024-09-11T05:27:11.557406+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:11.557406+0000","last_undegraded":"2024-09-11T05:27:11.557406+0000","last_fullsized":"2024-09-11T05:27:11.557406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20134,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558519+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:11.558519+0000","last_peered":"2024-09-11T05:27:11.558519+0000","last_clean":"2024-09-11T05:27:11.558519+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:11.558519+0000","last_undegraded":"2024-09-11T05:27:11.558519+0000","last_fullsized":"2024-09-11T05:27:11.558519+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26015,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:08.488957+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:27:08.488957+0000","last_peered":"2024-09-11T05:27:08.488957+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:27:08.488957+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-374}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":118,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558274+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:11.558274+0000","last_peered":"2024-09-11T05:27:11.558274+0000","last_clean":"2024-09-11T05:27:11.558274+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:11.558274+0000","last_undegraded":"2024-09-11T05:27:11.558274+0000","last_fullsized":"2024-09-11T05:27:11.558274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558569+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:11.558569+0000","last_peered":"2024-09-11T05:27:11.558569+0000","last_clean":"2024-09-11T05:27:11.558569+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:11.558569+0000","last_undegraded":"2024-09-11T05:27:11.558569+0000","last_fullsized":"2024-09-11T05:27:11.558569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19965,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.559063+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:11.559063+0000","last_peered":"2024-09-11T05:27:11.559063+0000","last_clean":"2024-09-11T05:27:11.559063+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:11.559063+0000","last_undegraded":"2024-09-11T05:27:11.559063+0000","last_fullsized":"2024-09-11T05:27:11.559063+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:11.557329+0000","last_peered":"2024-09-11T05:27:11.557329+0000","last_clean":"2024-09-11T05:27:11.557329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:11.557329+0000","last_undegraded":"2024-09-11T05:27:11.557329+0000","last_fullsized":"2024-09-11T05:27:11.557329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557732+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:11.557732+0000","last_peered":"2024-09-11T05:27:11.557732+0000","last_clean":"2024-09-11T05:27:11.557732+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:11.557732+0000","last_undegraded":"2024-09-11T05:27:11.557732+0000","last_fullsized":"2024-09-11T05:27:11.557732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19769,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558294+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:11.558294+0000","last_peered":"2024-09-11T05:27:11.558294+0000","last_clean":"2024-09-11T05:27:11.558294+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:11.558294+0000","last_undegraded":"2024-09-11T05:27:11.558294+0000","last_fullsized":"2024-09-11T05:27:11.558294+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35016,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100696064,"data_stored":100678669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479256,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892584,"statfs":{"total":96636764160,"available":96146006016,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215279,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:12.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T05:27:12.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19672", 'reported_seq': 26015, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:08.488957+0000', 'last_change': '2024-09-11T05:26:41.466628+0000', 'last_active': '2024-09-11T05:27:08.488957+0000', 'last_peered': '2024-09-11T05:27:08.488957+0000', 'last_clean': '2024-09-11T05:14:12.126790+0000', 'last_became_active': '2024-09-11T05:14:14.319486+0000', 'last_became_peered': '2024-09-11T05:14:14.319486+0000', 'last_unstale': '2024-09-11T05:27:08.488957+0000', 'last_undegraded': '2024-09-11T05:14:13.887884+0000', 'last_fullsized': '2024-09-11T05:14:13.742360+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3300", 'last_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:09.784433+0000', 'objects_scrubbed': 3239, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1595, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 374, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16425, 'num_write_kb': 26372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3070, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -374}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:12.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:12.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20312, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530394+0000', 'last_change': '2024-09-11T05:14:14.337148+0000', 'last_active': '2024-09-11T05:26:42.530394+0000', 'last_peered': '2024-09-11T05:26:42.530394+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:26:42.530394+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:12.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:12.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19824, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:26:42.530445+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:26:42.530445+0000', 'last_peered': '2024-09-11T05:26:42.530445+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:26:42.530445+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:12.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:13.257 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:13.257 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7496/6 objects degraded (124933.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":911,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1426386944,"bytes_avail":288483905536,"bytes_total":289910292480,"degraded_objects":7496,"degraded_total":6,"degraded_ratio":1249.3333333333333,"recovering_objects_per_sec":108,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1084,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:13.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:13.281 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:13.633 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:13.633 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:13.651 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-11T05:27:12.543278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35017,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1380476,"kb_used_data":99836,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281735044,"statfs":{"total":289910292480,"available":288496685056,"internally_reserved":0,"allocated":102232064,"data_stored":101194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-589,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":589,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.716887"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.615300+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:26:42.615300+0000","last_peered":"2024-09-11T05:26:42.615300+0000","last_clean":"2024-09-11T05:26:42.615300+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:26:42.615300+0000","last_undegraded":"2024-09-11T05:26:42.615300+0000","last_fullsized":"2024-09-11T05:26:42.615300+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20039,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887764+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:26:42.887764+0000","last_peered":"2024-09-11T05:26:42.887764+0000","last_clean":"2024-09-11T05:26:42.887764+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:26:42.887764+0000","last_undegraded":"2024-09-11T05:26:42.887764+0000","last_fullsized":"2024-09-11T05:26:42.887764+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26883,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558830+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:11.558830+0000","last_peered":"2024-09-11T05:27:11.558830+0000","last_clean":"2024-09-11T05:27:11.558830+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:11.558830+0000","last_undegraded":"2024-09-11T05:27:11.558830+0000","last_fullsized":"2024-09-11T05:27:11.558830+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20068,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887751+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:26:42.887751+0000","last_peered":"2024-09-11T05:26:42.887751+0000","last_clean":"2024-09-11T05:26:42.887751+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:26:42.887751+0000","last_undegraded":"2024-09-11T05:26:42.887751+0000","last_fullsized":"2024-09-11T05:26:42.887751+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20335,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.530141+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:26:42.530141+0000","last_peered":"2024-09-11T05:26:42.530141+0000","last_clean":"2024-09-11T05:26:42.530141+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:26:42.530141+0000","last_undegraded":"2024-09-11T05:26:42.530141+0000","last_fullsized":"2024-09-11T05:26:42.530141+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27419,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558483+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:11.558483+0000","last_peered":"2024-09-11T05:27:11.558483+0000","last_clean":"2024-09-11T05:27:11.558483+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:11.558483+0000","last_undegraded":"2024-09-11T05:27:11.558483+0000","last_fullsized":"2024-09-11T05:27:11.558483+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22670,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557999+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:11.557999+0000","last_peered":"2024-09-11T05:27:11.557999+0000","last_clean":"2024-09-11T05:27:11.557999+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:11.557999+0000","last_undegraded":"2024-09-11T05:27:11.557999+0000","last_fullsized":"2024-09-11T05:27:11.557999+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20045,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529850+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:26:42.529850+0000","last_peered":"2024-09-11T05:26:42.529850+0000","last_clean":"2024-09-11T05:26:42.529850+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:26:42.529850+0000","last_undegraded":"2024-09-11T05:26:42.529850+0000","last_fullsized":"2024-09-11T05:26:42.529850+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557406+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:11.557406+0000","last_peered":"2024-09-11T05:27:11.557406+0000","last_clean":"2024-09-11T05:27:11.557406+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:11.557406+0000","last_undegraded":"2024-09-11T05:27:11.557406+0000","last_fullsized":"2024-09-11T05:27:11.557406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20134,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558519+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:11.558519+0000","last_peered":"2024-09-11T05:27:11.558519+0000","last_clean":"2024-09-11T05:27:11.558519+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:11.558519+0000","last_undegraded":"2024-09-11T05:27:11.558519+0000","last_fullsized":"2024-09-11T05:27:11.558519+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887605+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:26:42.887605+0000","last_peered":"2024-09-11T05:26:42.887605+0000","last_clean":"2024-09-11T05:26:42.887605+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:26:42.887605+0000","last_undegraded":"2024-09-11T05:26:42.887605+0000","last_fullsized":"2024-09-11T05:26:42.887605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26015,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:08.488957+0000","last_change":"2024-09-11T05:26:41.466628+0000","last_active":"2024-09-11T05:27:08.488957+0000","last_peered":"2024-09-11T05:27:08.488957+0000","last_clean":"2024-09-11T05:14:12.126790+0000","last_became_active":"2024-09-11T05:14:14.319486+0000","last_became_peered":"2024-09-11T05:14:14.319486+0000","last_unstale":"2024-09-11T05:27:08.488957+0000","last_undegraded":"2024-09-11T05:14:13.887884+0000","last_fullsized":"2024-09-11T05:14:13.742360+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3070,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-374}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.529688+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:26:42.529688+0000","last_peered":"2024-09-11T05:26:42.529688+0000","last_clean":"2024-09-11T05:26:42.529688+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:26:42.529688+0000","last_undegraded":"2024-09-11T05:26:42.529688+0000","last_fullsized":"2024-09-11T05:26:42.529688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27243,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887931+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:26:42.887931+0000","last_peered":"2024-09-11T05:26:42.887931+0000","last_clean":"2024-09-11T05:26:42.887931+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:26:42.887931+0000","last_undegraded":"2024-09-11T05:26:42.887931+0000","last_fullsized":"2024-09-11T05:26:42.887931+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":118,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558274+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:11.558274+0000","last_peered":"2024-09-11T05:27:11.558274+0000","last_clean":"2024-09-11T05:27:11.558274+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:11.558274+0000","last_undegraded":"2024-09-11T05:27:11.558274+0000","last_fullsized":"2024-09-11T05:27:11.558274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26830,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887815+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:26:42.887815+0000","last_peered":"2024-09-11T05:26:42.887815+0000","last_clean":"2024-09-11T05:26:42.887815+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:26:42.887815+0000","last_undegraded":"2024-09-11T05:26:42.887815+0000","last_fullsized":"2024-09-11T05:26:42.887815+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20312,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530394+0000","last_change":"2024-09-11T05:14:14.337148+0000","last_active":"2024-09-11T05:26:42.530394+0000","last_peered":"2024-09-11T05:26:42.530394+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:26:42.530394+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558569+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:11.558569+0000","last_peered":"2024-09-11T05:27:11.558569+0000","last_clean":"2024-09-11T05:27:11.558569+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:11.558569+0000","last_undegraded":"2024-09-11T05:27:11.558569+0000","last_fullsized":"2024-09-11T05:27:11.558569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19824,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:26:42.530445+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:26:42.530445+0000","last_peered":"2024-09-11T05:26:42.530445+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:26:42.530445+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T05:26:42.887599+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:26:42.887599+0000","last_peered":"2024-09-11T05:26:42.887599+0000","last_clean":"2024-09-11T05:26:42.887599+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:26:42.887599+0000","last_undegraded":"2024-09-11T05:26:42.887599+0000","last_fullsized":"2024-09-11T05:26:42.887599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19965,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.559063+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:11.559063+0000","last_peered":"2024-09-11T05:27:11.559063+0000","last_clean":"2024-09-11T05:27:11.559063+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:11.559063+0000","last_undegraded":"2024-09-11T05:27:11.559063+0000","last_fullsized":"2024-09-11T05:27:11.559063+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:11.557329+0000","last_peered":"2024-09-11T05:27:11.557329+0000","last_clean":"2024-09-11T05:27:11.557329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:11.557329+0000","last_undegraded":"2024-09-11T05:27:11.557329+0000","last_fullsized":"2024-09-11T05:27:11.557329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557732+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:11.557732+0000","last_peered":"2024-09-11T05:27:11.557732+0000","last_clean":"2024-09-11T05:27:11.557732+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:11.557732+0000","last_undegraded":"2024-09-11T05:27:11.557732+0000","last_fullsized":"2024-09-11T05:27:11.557732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19769,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558294+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:11.558294+0000","last_peered":"2024-09-11T05:27:11.558294+0000","last_clean":"2024-09-11T05:27:11.558294+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:11.558294+0000","last_undegraded":"2024-09-11T05:27:11.558294+0000","last_fullsized":"2024-09-11T05:27:11.558294+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35016,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100696064,"data_stored":100678669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389576,"kb_used_data":42736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982264,"statfs":{"total":96636764160,"available":96237838336,"internally_reserved":0,"allocated":43761664,"data_stored":43425329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479256,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892584,"statfs":{"total":96636764160,"available":96146006016,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215279,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511644,"kb_used_data":56580,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860196,"statfs":{"total":96636764160,"available":96112840704,"internally_reserved":0,"allocated":57937920,"data_stored":57594029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43253760,"data_stored":43253760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:16.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:17.022 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:17.023 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:17.043 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-11T05:27:15.829353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35652,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1380496,"kb_used_data":99860,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281735024,"statfs":{"total":289910292480,"available":288496664576,"internally_reserved":0,"allocated":102256640,"data_stored":101202350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002050"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26883,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558830+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:11.558830+0000","last_peered":"2024-09-11T05:27:11.558830+0000","last_clean":"2024-09-11T05:27:11.558830+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:11.558830+0000","last_undegraded":"2024-09-11T05:27:11.558830+0000","last_fullsized":"2024-09-11T05:27:11.558830+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27419,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558483+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:11.558483+0000","last_peered":"2024-09-11T05:27:11.558483+0000","last_clean":"2024-09-11T05:27:11.558483+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:11.558483+0000","last_undegraded":"2024-09-11T05:27:11.558483+0000","last_fullsized":"2024-09-11T05:27:11.558483+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22670,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557999+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:11.557999+0000","last_peered":"2024-09-11T05:27:11.557999+0000","last_clean":"2024-09-11T05:27:11.557999+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:11.557999+0000","last_undegraded":"2024-09-11T05:27:11.557999+0000","last_fullsized":"2024-09-11T05:27:11.557999+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557406+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:11.557406+0000","last_peered":"2024-09-11T05:27:11.557406+0000","last_clean":"2024-09-11T05:27:11.557406+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:11.557406+0000","last_undegraded":"2024-09-11T05:27:11.557406+0000","last_fullsized":"2024-09-11T05:27:11.557406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20134,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558519+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:11.558519+0000","last_peered":"2024-09-11T05:27:11.558519+0000","last_clean":"2024-09-11T05:27:11.558519+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:11.558519+0000","last_undegraded":"2024-09-11T05:27:11.558519+0000","last_fullsized":"2024-09-11T05:27:11.558519+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":118,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558274+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:11.558274+0000","last_peered":"2024-09-11T05:27:11.558274+0000","last_clean":"2024-09-11T05:27:11.558274+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:11.558274+0000","last_undegraded":"2024-09-11T05:27:11.558274+0000","last_fullsized":"2024-09-11T05:27:11.558274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20839,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:13.493866+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:13.493866+0000","last_peered":"2024-09-11T05:27:13.493866+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:13.493866+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558569+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:11.558569+0000","last_peered":"2024-09-11T05:27:11.558569+0000","last_clean":"2024-09-11T05:27:11.558569+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:11.558569+0000","last_undegraded":"2024-09-11T05:27:11.558569+0000","last_fullsized":"2024-09-11T05:27:11.558569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19965,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.559063+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:11.559063+0000","last_peered":"2024-09-11T05:27:11.559063+0000","last_clean":"2024-09-11T05:27:11.559063+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:11.559063+0000","last_undegraded":"2024-09-11T05:27:11.559063+0000","last_fullsized":"2024-09-11T05:27:11.559063+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:11.557329+0000","last_peered":"2024-09-11T05:27:11.557329+0000","last_clean":"2024-09-11T05:27:11.557329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:11.557329+0000","last_undegraded":"2024-09-11T05:27:11.557329+0000","last_fullsized":"2024-09-11T05:27:11.557329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557732+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:11.557732+0000","last_peered":"2024-09-11T05:27:11.557732+0000","last_clean":"2024-09-11T05:27:11.557732+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:11.557732+0000","last_undegraded":"2024-09-11T05:27:11.557732+0000","last_fullsized":"2024-09-11T05:27:11.557732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19769,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558294+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:11.558294+0000","last_peered":"2024-09-11T05:27:11.558294+0000","last_clean":"2024-09-11T05:27:11.558294+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:11.558294+0000","last_undegraded":"2024-09-11T05:27:11.558294+0000","last_fullsized":"2024-09-11T05:27:11.558294+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35651,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93081600,"data_stored":93064205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389580,"kb_used_data":42744,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982260,"statfs":{"total":96636764160,"available":96237834240,"internally_reserved":0,"allocated":43769856,"data_stored":43428109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479256,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892584,"statfs":{"total":96636764160,"available":96146006016,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":35639296,"data_stored":35639296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:17.044 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:17.375 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:17.376 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:17.396 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-11T05:27:15.829353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35652,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1380496,"kb_used_data":99860,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281735024,"statfs":{"total":289910292480,"available":288496664576,"internally_reserved":0,"allocated":102256640,"data_stored":101202350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002050"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26883,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558830+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:11.558830+0000","last_peered":"2024-09-11T05:27:11.558830+0000","last_clean":"2024-09-11T05:27:11.558830+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:11.558830+0000","last_undegraded":"2024-09-11T05:27:11.558830+0000","last_fullsized":"2024-09-11T05:27:11.558830+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27419,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558483+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:11.558483+0000","last_peered":"2024-09-11T05:27:11.558483+0000","last_clean":"2024-09-11T05:27:11.558483+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:11.558483+0000","last_undegraded":"2024-09-11T05:27:11.558483+0000","last_fullsized":"2024-09-11T05:27:11.558483+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22670,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557999+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:11.557999+0000","last_peered":"2024-09-11T05:27:11.557999+0000","last_clean":"2024-09-11T05:27:11.557999+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:11.557999+0000","last_undegraded":"2024-09-11T05:27:11.557999+0000","last_fullsized":"2024-09-11T05:27:11.557999+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557406+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:11.557406+0000","last_peered":"2024-09-11T05:27:11.557406+0000","last_clean":"2024-09-11T05:27:11.557406+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:11.557406+0000","last_undegraded":"2024-09-11T05:27:11.557406+0000","last_fullsized":"2024-09-11T05:27:11.557406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20134,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558519+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:11.558519+0000","last_peered":"2024-09-11T05:27:11.558519+0000","last_clean":"2024-09-11T05:27:11.558519+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:11.558519+0000","last_undegraded":"2024-09-11T05:27:11.558519+0000","last_fullsized":"2024-09-11T05:27:11.558519+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":118,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558274+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:11.558274+0000","last_peered":"2024-09-11T05:27:11.558274+0000","last_clean":"2024-09-11T05:27:11.558274+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:11.558274+0000","last_undegraded":"2024-09-11T05:27:11.558274+0000","last_fullsized":"2024-09-11T05:27:11.558274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20839,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:13.493866+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:13.493866+0000","last_peered":"2024-09-11T05:27:13.493866+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:13.493866+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558569+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:11.558569+0000","last_peered":"2024-09-11T05:27:11.558569+0000","last_clean":"2024-09-11T05:27:11.558569+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:11.558569+0000","last_undegraded":"2024-09-11T05:27:11.558569+0000","last_fullsized":"2024-09-11T05:27:11.558569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19965,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.559063+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:11.559063+0000","last_peered":"2024-09-11T05:27:11.559063+0000","last_clean":"2024-09-11T05:27:11.559063+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:11.559063+0000","last_undegraded":"2024-09-11T05:27:11.559063+0000","last_fullsized":"2024-09-11T05:27:11.559063+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:11.557329+0000","last_peered":"2024-09-11T05:27:11.557329+0000","last_clean":"2024-09-11T05:27:11.557329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:11.557329+0000","last_undegraded":"2024-09-11T05:27:11.557329+0000","last_fullsized":"2024-09-11T05:27:11.557329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.557732+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:11.557732+0000","last_peered":"2024-09-11T05:27:11.557732+0000","last_clean":"2024-09-11T05:27:11.557732+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:11.557732+0000","last_undegraded":"2024-09-11T05:27:11.557732+0000","last_fullsized":"2024-09-11T05:27:11.557732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19769,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-11T05:27:11.558294+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:11.558294+0000","last_peered":"2024-09-11T05:27:11.558294+0000","last_clean":"2024-09-11T05:27:11.558294+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:11.558294+0000","last_undegraded":"2024-09-11T05:27:11.558294+0000","last_fullsized":"2024-09-11T05:27:11.558294+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35651,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93081600,"data_stored":93064205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389580,"kb_used_data":42744,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982260,"statfs":{"total":96636764160,"available":96237834240,"internally_reserved":0,"allocated":43769856,"data_stored":43428109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479256,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892584,"statfs":{"total":96636764160,"available":96146006016,"internally_reserved":0,"allocated":532480,"data_stored":174673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":35639296,"data_stored":35639296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:17.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:17.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 20839, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:13.493866+0000', 'last_change': '2024-09-11T05:27:11.484857+0000', 'last_active': '2024-09-11T05:27:13.493866+0000', 'last_peered': '2024-09-11T05:27:13.493866+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:27:13.493866+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3363, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 261, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:17.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:17.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:17.398 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:17.814 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:17.814 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6861/6 objects degraded (114350.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":915,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1413627904,"bytes_avail":288496664576,"bytes_total":289910292480,"degraded_objects":6861,"degraded_total":6,"degraded_ratio":1143.5,"recovering_objects_per_sec":152,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1224,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:17.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:17.838 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:18.177 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:18.178 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:18.197 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-11T05:27:17.830024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35652,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384668,"kb_used_data":99872,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281730852,"statfs":{"total":289910292480,"available":288492392448,"internally_reserved":0,"allocated":102268928,"data_stored":101207272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002312"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20839,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:13.493866+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:13.493866+0000","last_peered":"2024-09-11T05:27:13.493866+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:13.493866+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35651,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93081600,"data_stored":93064205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389580,"kb_used_data":42744,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93982260,"statfs":{"total":96636764160,"available":96237834240,"internally_reserved":0,"allocated":43769856,"data_stored":43428109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":35639296,"data_stored":35639296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:21.200 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:21.573 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:21.594 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-09-11T05:27:19.830484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36367,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370912,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281744608,"statfs":{"total":289910292480,"available":288506478592,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1350,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.290464"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":22269,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:18.503375+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:18.503375+0000","last_peered":"2024-09-11T05:27:18.503375+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:18.503375+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36366,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:21.595 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:21.922 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:21.941 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-11T05:27:21.830874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36367,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370912,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281744608,"statfs":{"total":289910292480,"available":288506478592,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1350,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002510"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":22269,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:18.503375+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:18.503375+0000","last_peered":"2024-09-11T05:27:18.503375+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:18.503375+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36366,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:21.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:21.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 22269, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:18.503375+0000', 'last_change': '2024-09-11T05:27:11.484857+0000', 'last_active': '2024-09-11T05:27:18.503375+0000', 'last_peered': '2024-09-11T05:27:18.503375+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:27:18.503375+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2648, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 976, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:21.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:21.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:21.943 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:22.377 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:22.377 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6146/6 objects degraded (102433.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":920,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1403813888,"bytes_avail":288506478592,"bytes_total":289910292480,"degraded_objects":6146,"degraded_total":6,"degraded_ratio":1024.3333333333333,"recovering_objects_per_sec":134,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1350,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:22.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:22.397 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:22.749 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:22.749 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:22.768 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-11T05:27:21.830874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36367,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370912,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281744608,"statfs":{"total":289910292480,"available":288506478592,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1350,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002510"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":22269,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:18.503375+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:18.503375+0000","last_peered":"2024-09-11T05:27:18.503375+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:18.503375+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36366,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:25.771 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:26.112 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:26.113 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:26.131 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-11T05:27:25.831915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37016,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1379232,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281736288,"statfs":{"total":289910292480,"available":288497958912,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003001"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":23567,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:23.496336+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:23.496336+0000","last_peered":"2024-09-11T05:27:23.496336+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:23.496336+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37015,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491748,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880092,"statfs":{"total":96636764160,"available":96133214208,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215282,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:26.132 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:26.441 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:26.441 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:26.458 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-11T05:27:25.831915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37016,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1379232,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281736288,"statfs":{"total":289910292480,"available":288497958912,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003001"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":23567,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:23.496336+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:23.496336+0000","last_peered":"2024-09-11T05:27:23.496336+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:23.496336+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37015,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491748,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880092,"statfs":{"total":96636764160,"available":96133214208,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215282,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:26.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:26.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 23567, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:23.496336+0000', 'last_change': '2024-09-11T05:27:11.484857+0000', 'last_active': '2024-09-11T05:27:23.496336+0000', 'last_peered': '2024-09-11T05:27:23.496336+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:27:23.496336+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1999, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1625, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1999}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:26.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:26.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:26.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:26.820 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:26.821 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5497/6 objects degraded (91616.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":924,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1412333568,"bytes_avail":288497958912,"bytes_total":289910292480,"degraded_objects":5497,"degraded_total":6,"degraded_ratio":916.16666666666652,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:26.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:26.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:27.144 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:27.144 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:27.162 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-11T05:27:25.831915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37016,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1379232,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1293043,"kb_avail":281736288,"statfs":{"total":289910292480,"available":288497958912,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1324076033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003001"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":23567,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:23.496336+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:23.496336+0000","last_peered":"2024-09-11T05:27:23.496336+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:23.496336+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37015,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491748,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880092,"statfs":{"total":96636764160,"available":96133214208,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":502984363},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215282,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:30.164 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:30.490 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:30.491 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:30.508 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-11T05:27:29.833041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1366752,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281748768,"statfs":{"total":289910292480,"available":288510738432,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003016"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":24371,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:28.502802+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:28.502802+0000","last_peered":"2024-09-11T05:27:28.502802+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:28.502802+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2027,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37417,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215283,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:30.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:30.820 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:30.820 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:30.838 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-11T05:27:29.833041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1366752,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281748768,"statfs":{"total":289910292480,"available":288510738432,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003016"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":24371,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:28.502802+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:28.502802+0000","last_peered":"2024-09-11T05:27:28.502802+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:28.502802+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2027,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37417,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215283,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 24371, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:28.502802+0000', 'last_change': '2024-09-11T05:27:11.484857+0000', 'last_active': '2024-09-11T05:27:28.502802+0000', 'last_peered': '2024-09-11T05:27:28.502802+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:27:28.502802+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1597, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2027, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1597}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:30.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:31.238 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:31.238 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5095/6 objects degraded (84916.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":929,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1399554048,"bytes_avail":288510738432,"bytes_total":289910292480,"degraded_objects":5095,"degraded_total":6,"degraded_ratio":849.16666666666652,"recovering_objects_per_sec":147,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1766,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:31.257 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:31.258 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:31.585 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:31.585 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:31.606 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-11T05:27:29.833041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37418,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1366752,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281748768,"statfs":{"total":289910292480,"available":288510738432,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1311296513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1766,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003016"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544910+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544910+0000","last_peered":"2024-09-11T05:27:12.544910+0000","last_clean":"2024-09-11T05:27:12.544910+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544910+0000","last_undegraded":"2024-09-11T05:27:12.544910+0000","last_fullsized":"2024-09-11T05:27:12.544910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":24371,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:28.502802+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:28.502802+0000","last_peered":"2024-09-11T05:27:28.502802+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:28.502802+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2027,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37417,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":490204843},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215283,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:34.609 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:34.966 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:34.984 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-11T05:27:33.834368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37852,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370912,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281744608,"statfs":{"total":289910292480,"available":288506478592,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1485,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003494"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":25239,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:33.501259+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:33.501259+0000","last_peered":"2024-09-11T05:27:33.501259+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:33.501259+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37851,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215284,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:34.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:35.302 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:35.302 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:35.320 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-11T05:27:33.834368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37852,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370912,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281744608,"statfs":{"total":289910292480,"available":288506478592,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1485,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003494"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":25239,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:33.501259+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:33.501259+0000","last_peered":"2024-09-11T05:27:33.501259+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:33.501259+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37851,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215284,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:35.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:35.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 25239, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:33.501259+0000', 'last_change': '2024-09-11T05:27:11.484857+0000', 'last_active': '2024-09-11T05:27:33.501259+0000', 'last_peered': '2024-09-11T05:27:33.501259+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:27:33.501259+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1163, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2461, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:35.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:35.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:35.322 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:35.724 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:35.725 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4661/6 objects degraded (77683.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":933,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1403813888,"bytes_avail":288506478592,"bytes_total":289910292480,"degraded_objects":4661,"degraded_total":6,"degraded_ratio":776.83333333333348,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1485,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:35.744 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:35.744 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:36.062 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:36.084 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-09-11T05:27:35.835145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37852,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1370912,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281744608,"statfs":{"total":289910292480,"available":288506478592,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1315556353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":836,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003882"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":25239,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:33.501259+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:33.501259+0000","last_peered":"2024-09-11T05:27:33.501259+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:33.501259+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37851,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":494464683},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215284,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:39.086 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:39.437 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:39.438 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:39.457 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-09-11T05:27:37.835486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37852,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1393120,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281722400,"statfs":{"total":289910292480,"available":288483737600,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":836,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":25239,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:33.501259+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:33.501259+0000","last_peered":"2024-09-11T05:27:33.501259+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:33.501259+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37851,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505636,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866204,"statfs":{"total":96636764160,"available":96118992896,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215284,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:39.458 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:39.766 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:39.767 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:39.786 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-09-11T05:27:37.835486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37852,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1393120,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281722400,"statfs":{"total":289910292480,"available":288483737600,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":836,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":25239,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:33.501259+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:33.501259+0000","last_peered":"2024-09-11T05:27:33.501259+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:33.501259+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37851,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505636,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866204,"statfs":{"total":96636764160,"available":96118992896,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215284,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:39.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:39.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 25239, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:33.501259+0000', 'last_change': '2024-09-11T05:27:11.484857+0000', 'last_active': '2024-09-11T05:27:33.501259+0000', 'last_peered': '2024-09-11T05:27:33.501259+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:27:33.501259+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1163, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2461, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:39.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:39.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:39.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:40.163 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:40.164 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4164/6 objects degraded (69400.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":937,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1426554880,"bytes_avail":288483737600,"bytes_total":289910292480,"degraded_objects":4164,"degraded_total":6,"degraded_ratio":694,"recovering_objects_per_sec":111,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1333,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:40.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:40.184 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:40.505 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:40.505 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:40.524 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-11T05:27:39.836204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38349,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1393120,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281722400,"statfs":{"total":289910292480,"available":288483737600,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1333,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003806"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":26233,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:38.507637+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:38.507637+0000","last_peered":"2024-09-11T05:27:38.507637+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:38.507637+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2958,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-666}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38348,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505636,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866204,"statfs":{"total":96636764160,"available":96118992896,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215285,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:43.526 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:43.874 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:43.874 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:43.896 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2024-09-11T05:27:43.837256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38993,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397280,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281718240,"statfs":{"total":289910292480,"available":288479477760,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1575,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003576"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":27521,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:43.504507+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:43.504507+0000","last_peered":"2024-09-11T05:27:43.504507+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:43.504507+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-22}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38992,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509796,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862044,"statfs":{"total":96636764160,"available":96114733056,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:43.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:44.215 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:44.216 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:44.234 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2024-09-11T05:27:43.917735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38993,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397280,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281718240,"statfs":{"total":289910292480,"available":288479477760,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.083367"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":27521,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:43.504507+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:43.504507+0000","last_peered":"2024-09-11T05:27:43.504507+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:43.504507+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-22}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38992,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509796,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862044,"statfs":{"total":96636764160,"available":96114733056,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:44.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T05:27:44.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20144", 'reported_seq': 27521, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:43.504507+0000', 'last_change': '2024-09-11T05:27:11.484857+0000', 'last_active': '2024-09-11T05:27:43.504507+0000', 'last_peered': '2024-09-11T05:27:43.504507+0000', 'last_clean': '2024-09-11T05:14:12.129970+0000', 'last_became_active': '2024-09-11T05:14:14.319973+0000', 'last_became_peered': '2024-09-11T05:14:14.319973+0000', 'last_unstale': '2024-09-11T05:27:43.504507+0000', 'last_undegraded': '2024-09-11T05:14:13.905718+0000', 'last_fullsized': '2024-09-11T05:14:13.743271+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3023", 'last_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:01.854947+0000', 'objects_scrubbed': 2965, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1631, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 22, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17177, 'num_write_kb': 28424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3602, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -22}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:44.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:44.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:44.236 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:44.595 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:44.595 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3520/6 objects degraded (58666.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":942,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1430814720,"bytes_avail":288479477760,"bytes_total":289910292480,"degraded_objects":3520,"degraded_total":6,"degraded_ratio":586.66666666666663,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1575,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:44.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:44.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:44.915 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:44.940 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2024-09-11T05:27:43.917735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38993,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397280,"kb_used_data":86112,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281718240,"statfs":{"total":289910292480,"available":288479477760,"internally_reserved":0,"allocated":88178688,"data_stored":87111599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.083367"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26885,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547238+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:12.547238+0000","last_peered":"2024-09-11T05:27:12.547238+0000","last_clean":"2024-09-11T05:27:12.547238+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:12.547238+0000","last_undegraded":"2024-09-11T05:27:12.547238+0000","last_fullsized":"2024-09-11T05:27:12.547238+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27421,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547275+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:12.547275+0000","last_peered":"2024-09-11T05:27:12.547275+0000","last_clean":"2024-09-11T05:27:12.547275+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:12.547275+0000","last_undegraded":"2024-09-11T05:27:12.547275+0000","last_fullsized":"2024-09-11T05:27:12.547275+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22672,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547153+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:12.547153+0000","last_peered":"2024-09-11T05:27:12.547153+0000","last_clean":"2024-09-11T05:27:12.547153+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:12.547153+0000","last_undegraded":"2024-09-11T05:27:12.547153+0000","last_fullsized":"2024-09-11T05:27:12.547153+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547257+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:12.547257+0000","last_peered":"2024-09-11T05:27:12.547257+0000","last_clean":"2024-09-11T05:27:12.547257+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:12.547257+0000","last_undegraded":"2024-09-11T05:27:12.547257+0000","last_fullsized":"2024-09-11T05:27:12.547257+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546910+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:12.546910+0000","last_peered":"2024-09-11T05:27:12.546910+0000","last_clean":"2024-09-11T05:27:12.546910+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:12.546910+0000","last_undegraded":"2024-09-11T05:27:12.546910+0000","last_fullsized":"2024-09-11T05:27:12.546910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":27521,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:43.504507+0000","last_change":"2024-09-11T05:27:11.484857+0000","last_active":"2024-09-11T05:27:43.504507+0000","last_peered":"2024-09-11T05:27:43.504507+0000","last_clean":"2024-09-11T05:14:12.129970+0000","last_became_active":"2024-09-11T05:14:14.319973+0000","last_became_peered":"2024-09-11T05:14:14.319973+0000","last_unstale":"2024-09-11T05:27:43.504507+0000","last_undegraded":"2024-09-11T05:14:13.905718+0000","last_fullsized":"2024-09-11T05:14:13.743271+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-22}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547143+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:12.547143+0000","last_peered":"2024-09-11T05:27:12.547143+0000","last_clean":"2024-09-11T05:27:12.547143+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:12.547143+0000","last_undegraded":"2024-09-11T05:27:12.547143+0000","last_fullsized":"2024-09-11T05:27:12.547143+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547340+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:12.547340+0000","last_peered":"2024-09-11T05:27:12.547340+0000","last_clean":"2024-09-11T05:27:12.547340+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:12.547340+0000","last_undegraded":"2024-09-11T05:27:12.547340+0000","last_fullsized":"2024-09-11T05:27:12.547340+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547436+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:12.547436+0000","last_peered":"2024-09-11T05:27:12.547436+0000","last_clean":"2024-09-11T05:27:12.547436+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:12.547436+0000","last_undegraded":"2024-09-11T05:27:12.547436+0000","last_fullsized":"2024-09-11T05:27:12.547436+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547414+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:12.547414+0000","last_peered":"2024-09-11T05:27:12.547414+0000","last_clean":"2024-09-11T05:27:12.547414+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:12.547414+0000","last_undegraded":"2024-09-11T05:27:12.547414+0000","last_fullsized":"2024-09-11T05:27:12.547414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19772,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.547094+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:12.547094+0000","last_peered":"2024-09-11T05:27:12.547094+0000","last_clean":"2024-09-11T05:27:12.547094+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:12.547094+0000","last_undegraded":"2024-09-11T05:27:12.547094+0000","last_fullsized":"2024-09-11T05:27:12.547094+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38992,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86597632,"data_stored":86580237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509796,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862044,"statfs":{"total":96636764160,"available":96114733056,"internally_reserved":0,"allocated":544768,"data_stored":179595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:47.942 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:48.326 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:48.326 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:48.352 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-11T05:27:47.838674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39015,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384804,"kb_used_data":86116,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281730716,"statfs":{"total":289910292480,"available":288492253184,"internally_reserved":0,"allocated":88182784,"data_stored":87115237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1329777665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002469"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39014,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86585344,"data_stored":86578386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:48.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:48.698 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:48.717 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-11T05:27:47.838674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39015,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384804,"kb_used_data":86116,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281730716,"statfs":{"total":289910292480,"available":288492253184,"internally_reserved":0,"allocated":88182784,"data_stored":87115237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1329777665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002469"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39014,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86585344,"data_stored":86578386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:48.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:48.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 19828, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:12.545665+0000', 'last_change': '2024-09-11T05:14:14.352843+0000', 'last_active': '2024-09-11T05:27:12.545665+0000', 'last_peered': '2024-09-11T05:27:12.545665+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:12.545665+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3498, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:48.719 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:49.119 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:49.119 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3498/6 objects degraded (58300.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":946,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1418039296,"bytes_avail":288492253184,"bytes_total":289910292480,"degraded_objects":3498,"degraded_total":6,"degraded_ratio":583,"recovering_objects_per_sec":83,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":666,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:49.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:49.141 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:49.467 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:49.490 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-11T05:27:47.838674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39015,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1384804,"kb_used_data":86116,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281730716,"statfs":{"total":289910292480,"available":288492253184,"internally_reserved":0,"allocated":88182784,"data_stored":87115237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1329777665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":666,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002469"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.546831+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:12.546831+0000","last_peered":"2024-09-11T05:27:12.546831+0000","last_clean":"2024-09-11T05:27:12.546831+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:12.546831+0000","last_undegraded":"2024-09-11T05:27:12.546831+0000","last_fullsized":"2024-09-11T05:27:12.546831+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20043,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937347+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:12.937347+0000","last_peered":"2024-09-11T05:27:12.937347+0000","last_clean":"2024-09-11T05:27:12.937347+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:12.937347+0000","last_undegraded":"2024-09-11T05:27:12.937347+0000","last_fullsized":"2024-09-11T05:27:12.937347+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20072,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936995+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:12.936995+0000","last_peered":"2024-09-11T05:27:12.936995+0000","last_clean":"2024-09-11T05:27:12.936995+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:12.936995+0000","last_undegraded":"2024-09-11T05:27:12.936995+0000","last_fullsized":"2024-09-11T05:27:12.936995+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20339,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.545131+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:12.545131+0000","last_peered":"2024-09-11T05:27:12.545131+0000","last_clean":"2024-09-11T05:27:12.545131+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:12.545131+0000","last_undegraded":"2024-09-11T05:27:12.545131+0000","last_fullsized":"2024-09-11T05:27:12.545131+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20049,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544904+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:12.544904+0000","last_peered":"2024-09-11T05:27:12.544904+0000","last_clean":"2024-09-11T05:27:12.544904+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:12.544904+0000","last_undegraded":"2024-09-11T05:27:12.544904+0000","last_fullsized":"2024-09-11T05:27:12.544904+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936705+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:12.936705+0000","last_peered":"2024-09-11T05:27:12.936705+0000","last_clean":"2024-09-11T05:27:12.936705+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:12.936705+0000","last_undegraded":"2024-09-11T05:27:12.936705+0000","last_fullsized":"2024-09-11T05:27:12.936705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26774,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.608857+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:12.608857+0000","last_peered":"2024-09-11T05:27:12.608857+0000","last_clean":"2024-09-11T05:27:12.608857+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:12.608857+0000","last_undegraded":"2024-09-11T05:27:12.608857+0000","last_fullsized":"2024-09-11T05:27:12.608857+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.544995+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:12.544995+0000","last_peered":"2024-09-11T05:27:12.544995+0000","last_clean":"2024-09-11T05:27:12.544995+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:12.544995+0000","last_undegraded":"2024-09-11T05:27:12.544995+0000","last_fullsized":"2024-09-11T05:27:12.544995+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27247,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936792+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:12.936792+0000","last_peered":"2024-09-11T05:27:12.936792+0000","last_clean":"2024-09-11T05:27:12.936792+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:12.936792+0000","last_undegraded":"2024-09-11T05:27:12.936792+0000","last_fullsized":"2024-09-11T05:27:12.936792+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26834,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.936607+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:12.936607+0000","last_peered":"2024-09-11T05:27:12.936607+0000","last_clean":"2024-09-11T05:27:12.936607+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:12.936607+0000","last_undegraded":"2024-09-11T05:27:12.936607+0000","last_fullsized":"2024-09-11T05:27:12.936607+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19828,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:12.545665+0000","last_change":"2024-09-11T05:14:14.352843+0000","last_active":"2024-09-11T05:27:12.545665+0000","last_peered":"2024-09-11T05:27:12.545665+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:12.545665+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T05:27:12.937086+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:12.937086+0000","last_peered":"2024-09-11T05:27:12.937086+0000","last_clean":"2024-09-11T05:27:12.937086+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:12.937086+0000","last_undegraded":"2024-09-11T05:27:12.937086+0000","last_fullsized":"2024-09-11T05:27:12.937086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39014,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86585344,"data_stored":86578386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375824,"kb_used_data":28984,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93996016,"statfs":{"total":96636764160,"available":96251920384,"internally_reserved":0,"allocated":29679616,"data_stored":29332436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355135147},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511660,"kb_used_data":56596,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93860180,"statfs":{"total":96636764160,"available":96112824320,"internally_reserved":0,"allocated":57954304,"data_stored":57599568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":29155328,"data_stored":29155328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:52.493 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:52.862 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:52.863 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:52.885 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-11T05:27:51.839960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39550,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378660,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281736860,"statfs":{"total":289910292480,"available":288498544640,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002704"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936217+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936217+0000","last_peered":"2024-09-11T05:27:44.936217+0000","last_clean":"2024-09-11T05:27:44.936217+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936217+0000","last_undegraded":"2024-09-11T05:27:44.936217+0000","last_fullsized":"2024-09-11T05:27:44.936217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27251,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936298+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936298+0000","last_peered":"2024-09-11T05:27:44.936298+0000","last_clean":"2024-09-11T05:27:44.936298+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936298+0000","last_undegraded":"2024-09-11T05:27:44.936298+0000","last_fullsized":"2024-09-11T05:27:44.936298+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26838,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935980+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.935980+0000","last_peered":"2024-09-11T05:27:44.935980+0000","last_clean":"2024-09-11T05:27:44.935980+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.935980+0000","last_undegraded":"2024-09-11T05:27:44.935980+0000","last_fullsized":"2024-09-11T05:27:44.935980+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":20903,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:48.504705+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:48.504705+0000","last_peered":"2024-09-11T05:27:48.504705+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:48.504705+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2963}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936636+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936636+0000","last_peered":"2024-09-11T05:27:44.936636+0000","last_clean":"2024-09-11T05:27:44.936636+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936636+0000","last_undegraded":"2024-09-11T05:27:44.936636+0000","last_fullsized":"2024-09-11T05:27:44.936636+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39549,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:52.887 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:53.212 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:53.212 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:53.233 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-11T05:27:51.839960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39550,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378660,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281736860,"statfs":{"total":289910292480,"available":288498544640,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002704"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936217+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936217+0000","last_peered":"2024-09-11T05:27:44.936217+0000","last_clean":"2024-09-11T05:27:44.936217+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936217+0000","last_undegraded":"2024-09-11T05:27:44.936217+0000","last_fullsized":"2024-09-11T05:27:44.936217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27251,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936298+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936298+0000","last_peered":"2024-09-11T05:27:44.936298+0000","last_clean":"2024-09-11T05:27:44.936298+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936298+0000","last_undegraded":"2024-09-11T05:27:44.936298+0000","last_fullsized":"2024-09-11T05:27:44.936298+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26838,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935980+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.935980+0000","last_peered":"2024-09-11T05:27:44.935980+0000","last_clean":"2024-09-11T05:27:44.935980+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.935980+0000","last_undegraded":"2024-09-11T05:27:44.935980+0000","last_fullsized":"2024-09-11T05:27:44.935980+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":20903,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:48.504705+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:48.504705+0000","last_peered":"2024-09-11T05:27:48.504705+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:48.504705+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2963}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936636+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936636+0000","last_peered":"2024-09-11T05:27:44.936636+0000","last_clean":"2024-09-11T05:27:44.936636+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936636+0000","last_undegraded":"2024-09-11T05:27:44.936636+0000","last_fullsized":"2024-09-11T05:27:44.936636+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39549,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:53.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:53.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 20903, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:48.504705+0000', 'last_change': '2024-09-11T05:27:43.674443+0000', 'last_active': '2024-09-11T05:27:48.504705+0000', 'last_peered': '2024-09-11T05:27:48.504705+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:48.504705+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2963, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 535, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:53.234 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:53.661 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:53.661 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2963/6 objects degraded (49383.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":951,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1411747840,"bytes_avail":288498544640,"bytes_total":289910292480,"degraded_objects":2963,"degraded_total":6,"degraded_ratio":493.83333333333326,"recovering_objects_per_sec":69,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":557,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:53.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:53.681 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:54.013 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:54.032 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-11T05:27:53.840449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40304,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1382820,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281732700,"statfs":{"total":289910292480,"available":288494284800,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.922713"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":22411,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:53.429647+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:53.429647+0000","last_peered":"2024-09-11T05:27:53.429647+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:53.429647+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40303,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986355,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501480,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870360,"statfs":{"total":96636764160,"available":96123248640,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215288,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:57.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:57.357 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:57.358 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:57.378 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-09-11T05:27:55.840919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40304,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1382820,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281732700,"statfs":{"total":289910292480,"available":288494284800,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.908803"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":22411,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:53.429647+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:53.429647+0000","last_peered":"2024-09-11T05:27:53.429647+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:53.429647+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40303,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986355,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501480,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870360,"statfs":{"total":96636764160,"available":96123248640,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215288,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:57.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:57.675 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:57.676 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:57.693 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-09-11T05:27:55.840919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40304,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1382820,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281732700,"statfs":{"total":289910292480,"available":288494284800,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.908803"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":22411,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:53.429647+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:53.429647+0000","last_peered":"2024-09-11T05:27:53.429647+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:53.429647+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40303,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986355,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501480,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870360,"statfs":{"total":96636764160,"available":96123248640,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215288,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:27:57.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:27:57.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 22411, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:53.429647+0000', 'last_change': '2024-09-11T05:27:43.674443+0000', 'last_active': '2024-09-11T05:27:53.429647+0000', 'last_peered': '2024-09-11T05:27:53.429647+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:53.429647+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2209, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1289, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:27:57.695 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:27:58.055 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:58.055 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2209/6 objects degraded (36816.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":955,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1416007680,"bytes_avail":288494284800,"bytes_total":289910292480,"degraded_objects":2209,"degraded_total":6,"degraded_ratio":368.16666666666674,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1311,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:27:58.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:27:58.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:27:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:27:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:27:58.399 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2024-09-11T05:27:57.841585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40304,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1386980,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281728540,"statfs":{"total":289910292480,"available":288490024960,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003669"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":22411,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:53.429647+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:53.429647+0000","last_peered":"2024-09-11T05:27:53.429647+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:53.429647+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40303,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986355,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659301,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215288,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:01.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:01.732 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:01.751 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":537,"stamp":"2024-09-11T05:27:59.842015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40733,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1386980,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281728540,"statfs":{"total":289910292480,"available":288490024960,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003341"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":23269,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:58.432287+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:58.432287+0000","last_peered":"2024-09-11T05:27:58.432287+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:58.432287+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40732,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986356,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659301,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215289,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:01.752 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:02.100 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:02.100 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:02.119 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":538,"stamp":"2024-09-11T05:28:01.842599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40733,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1386980,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281728540,"statfs":{"total":289910292480,"available":288490024960,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003320"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":23269,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:58.432287+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:58.432287+0000","last_peered":"2024-09-11T05:27:58.432287+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:58.432287+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40732,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986356,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659301,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215289,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:02.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:28:02.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 23269, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:27:58.432287+0000', 'last_change': '2024-09-11T05:27:43.674443+0000', 'last_active': '2024-09-11T05:27:58.432287+0000', 'last_peered': '2024-09-11T05:27:58.432287+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:27:58.432287+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1780, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1718, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1780}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:28:02.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:28:02.618 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:02.618 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1780/6 objects degraded (29666.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":960,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1420267520,"bytes_avail":288490024960,"bytes_total":289910292480,"degraded_objects":1780,"degraded_total":6,"degraded_ratio":296.66666666666669,"recovering_objects_per_sec":98,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1183,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:28:02.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:28:02.640 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:02.978 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":538,"stamp":"2024-09-11T05:28:01.842599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40733,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1386980,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281728540,"statfs":{"total":289910292480,"available":288490024960,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003320"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":23269,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:27:58.432287+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:27:58.432287+0000","last_peered":"2024-09-11T05:27:58.432287+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:27:58.432287+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40732,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986356,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659301,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505640,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866200,"statfs":{"total":96636764160,"available":96118988800,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215289,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:05.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:06.338 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2024-09-11T05:28:05.843787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41157,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1391140,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281724380,"statfs":{"total":289910292480,"available":288485765120,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1351076865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003338"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":24117,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:03.433562+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:03.433562+0000","last_peered":"2024-09-11T05:28:03.433562+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:03.433562+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41156,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986357,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509800,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862040,"statfs":{"total":96636764160,"available":96114728960,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215290,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:06.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:06.654 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:06.654 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:06.673 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2024-09-11T05:28:05.843787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41157,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1391140,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281724380,"statfs":{"total":289910292480,"available":288485765120,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1351076865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003338"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":24117,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:03.433562+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:03.433562+0000","last_peered":"2024-09-11T05:28:03.433562+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:03.433562+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41156,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986357,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509800,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862040,"statfs":{"total":96636764160,"available":96114728960,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215290,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:06.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:28:06.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 24117, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:28:03.433562+0000', 'last_change': '2024-09-11T05:27:43.674443+0000', 'last_active': '2024-09-11T05:28:03.433562+0000', 'last_peered': '2024-09-11T05:28:03.433562+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:28:03.433562+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1356, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2142, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1356}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:28:06.675 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:28:07.068 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:07.068 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1356/6 objects degraded (22600.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":964,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1424527360,"bytes_avail":288485765120,"bytes_total":289910292480,"degraded_objects":1356,"degraded_total":6,"degraded_ratio":226,"recovering_objects_per_sec":71,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T05:15:11.394904+0000","services":{}},"progress_events":{}} 2024-09-11T05:28:07.088 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:28:07.088 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:07.435 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:07.436 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:07.455 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2024-09-11T05:28:05.843787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41157,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1391140,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1319411,"kb_avail":281724380,"statfs":{"total":289910292480,"available":288485765120,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1351076865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003338"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":24117,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:03.433562+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:03.433562+0000","last_peered":"2024-09-11T05:28:03.433562+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:03.433562+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41156,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986357,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509800,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862040,"statfs":{"total":96636764160,"available":96114728960,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":521465515},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215290,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:10.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:10.800 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:10.800 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:10.819 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2024-09-11T05:28:09.845038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41630,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378660,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281736860,"statfs":{"total":289910292480,"available":288498544640,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003452"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":25064,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:08.438228+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:08.438228+0000","last_peered":"2024-09-11T05:28:08.438228+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:08.438228+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-883}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41629,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986358,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659303,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215291,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:10.820 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:11.131 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:11.131 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:11.149 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2024-09-11T05:28:09.845038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41630,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378660,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281736860,"statfs":{"total":289910292480,"available":288498544640,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003452"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":25064,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:08.438228+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:08.438228+0000","last_peered":"2024-09-11T05:28:08.438228+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:08.438228+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-883}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41629,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986358,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659303,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215291,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:11.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:28:11.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 25064, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:28:08.438228+0000', 'last_change': '2024-09-11T05:27:43.674443+0000', 'last_active': '2024-09-11T05:28:08.438228+0000', 'last_peered': '2024-09-11T05:28:08.438228+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:28:08.438228+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 883, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2615, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -883}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:28:11.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:28:11.522 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:11.522 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 883/6 objects degraded (14716.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":969,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1411747840,"bytes_avail":288498544640,"bytes_total":289910292480,"degraded_objects":883,"degraded_total":6,"degraded_ratio":147.16666666666666,"recovering_objects_per_sec":110,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T05:28:09.845175+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:28:11.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:28:11.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:11.858 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:11.878 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-09-11T05:28:11.845631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41630,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378660,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281736860,"statfs":{"total":289910292480,"available":288498544640,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1338297345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":897,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003616"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27425,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935908+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935908+0000","last_peered":"2024-09-11T05:27:44.935908+0000","last_clean":"2024-09-11T05:27:44.935908+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935908+0000","last_undegraded":"2024-09-11T05:27:44.935908+0000","last_fullsized":"2024-09-11T05:27:44.935908+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":25064,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:08.438228+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:08.438228+0000","last_peered":"2024-09-11T05:28:08.438228+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:08.438228+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-883}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41629,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986358,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659303,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497320,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874520,"statfs":{"total":96636764160,"available":96127508480,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":508685995},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215291,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:14.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:15.213 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:15.213 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:15.231 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-09-11T05:28:13.846063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42235,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1382820,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281732700,"statfs":{"total":289910292480,"available":288494284800,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1502,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1502,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27426,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935956+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935956+0000","last_peered":"2024-09-11T05:27:44.935956+0000","last_clean":"2024-09-11T05:27:44.935956+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935956+0000","last_undegraded":"2024-09-11T05:27:44.935956+0000","last_fullsized":"2024-09-11T05:27:44.935956+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":26274,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:13.436289+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:13.436289+0000","last_peered":"2024-09-11T05:28:13.436289+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:13.436289+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42234,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986359,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501480,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870360,"statfs":{"total":96636764160,"available":96123248640,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215292,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:15.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:15.556 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:15.577 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-09-11T05:28:15.549744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42235,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1382820,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281732700,"statfs":{"total":289910292480,"available":288494284800,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1078,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.706738"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27426,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935956+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935956+0000","last_peered":"2024-09-11T05:27:44.935956+0000","last_clean":"2024-09-11T05:27:44.935956+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935956+0000","last_undegraded":"2024-09-11T05:27:44.935956+0000","last_fullsized":"2024-09-11T05:27:44.935956+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":26274,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:13.436289+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:13.436289+0000","last_peered":"2024-09-11T05:28:13.436289+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:13.436289+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42234,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501480,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870360,"statfs":{"total":96636764160,"available":96123248640,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215292,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:15.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T05:28:15.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'19654", 'reported_seq': 26274, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T05:28:13.436289+0000', 'last_change': '2024-09-11T05:27:43.674443+0000', 'last_active': '2024-09-11T05:28:13.436289+0000', 'last_peered': '2024-09-11T05:28:13.436289+0000', 'last_clean': '2024-09-11T05:14:12.140466+0000', 'last_became_active': '2024-09-11T05:14:14.320348+0000', 'last_became_peered': '2024-09-11T05:14:14.320348+0000', 'last_unstale': '2024-09-11T05:28:13.436289+0000', 'last_undegraded': '2024-09-11T05:14:13.908501+0000', 'last_fullsized': '2024-09-11T05:14:13.761789+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 15, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3176", 'last_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T05:12:30.129133+0000', 'last_clean_scrub_stamp': '2024-09-11T05:14:07.390153+0000', 'objects_scrubbed': 3126, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1135, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 278, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16520, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3220, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T05:28:15.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T05:28:15.926 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:15.926 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 278/6 objects degraded (4633.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":973,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1726031651,"num_in_osds":3,"osd_in_since":1726031643,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1416007680,"bytes_avail":288494284800,"bytes_total":289910292480,"degraded_objects":278,"degraded_total":6,"degraded_ratio":46.333333333333343,"recovering_objects_per_sec":125,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1502,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T05:12:02:199900+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T05:28:09.845175+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T05:28:15.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T05:28:15.943 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:16.236 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:16.237 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:16.254 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":546,"stamp":"2024-09-11T05:28:15.846729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42235,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1382820,"kb_used_data":71656,"kb_used_omap":12,"kb_used_meta":1311091,"kb_avail":281732700,"statfs":{"total":289910292480,"available":288494284800,"internally_reserved":0,"allocated":73375744,"data_stored":72286407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1342557185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1078,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002941"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26889,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935563+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:27:44.935563+0000","last_peered":"2024-09-11T05:27:44.935563+0000","last_clean":"2024-09-11T05:27:44.935563+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:27:44.935563+0000","last_undegraded":"2024-09-11T05:27:44.935563+0000","last_fullsized":"2024-09-11T05:27:44.935563+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27426,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935956+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:27:44.935956+0000","last_peered":"2024-09-11T05:27:44.935956+0000","last_clean":"2024-09-11T05:27:44.935956+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:27:44.935956+0000","last_undegraded":"2024-09-11T05:27:44.935956+0000","last_fullsized":"2024-09-11T05:27:44.935956+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22676,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935718+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:27:44.935718+0000","last_peered":"2024-09-11T05:27:44.935718+0000","last_clean":"2024-09-11T05:27:44.935718+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:27:44.935718+0000","last_undegraded":"2024-09-11T05:27:44.935718+0000","last_fullsized":"2024-09-11T05:27:44.935718+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935510+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:27:44.935510+0000","last_peered":"2024-09-11T05:27:44.935510+0000","last_clean":"2024-09-11T05:27:44.935510+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:27:44.935510+0000","last_undegraded":"2024-09-11T05:27:44.935510+0000","last_fullsized":"2024-09-11T05:27:44.935510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20141,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935895+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:27:44.935895+0000","last_peered":"2024-09-11T05:27:44.935895+0000","last_clean":"2024-09-11T05:27:44.935895+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:27:44.935895+0000","last_undegraded":"2024-09-11T05:27:44.935895+0000","last_fullsized":"2024-09-11T05:27:44.935895+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935742+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:27:44.935742+0000","last_peered":"2024-09-11T05:27:44.935742+0000","last_clean":"2024-09-11T05:27:44.935742+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:27:44.935742+0000","last_undegraded":"2024-09-11T05:27:44.935742+0000","last_fullsized":"2024-09-11T05:27:44.935742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20296,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:45.068518+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:27:45.068518+0000","last_peered":"2024-09-11T05:27:45.068518+0000","last_clean":"2024-09-11T05:27:45.068518+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:27:45.068518+0000","last_undegraded":"2024-09-11T05:27:45.068518+0000","last_fullsized":"2024-09-11T05:27:45.068518+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936097+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:27:44.936097+0000","last_peered":"2024-09-11T05:27:44.936097+0000","last_clean":"2024-09-11T05:27:44.936097+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:27:44.936097+0000","last_undegraded":"2024-09-11T05:27:44.936097+0000","last_fullsized":"2024-09-11T05:27:44.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":26274,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T05:28:13.436289+0000","last_change":"2024-09-11T05:27:43.674443+0000","last_active":"2024-09-11T05:28:13.436289+0000","last_peered":"2024-09-11T05:28:13.436289+0000","last_clean":"2024-09-11T05:14:12.140466+0000","last_became_active":"2024-09-11T05:14:14.320348+0000","last_became_peered":"2024-09-11T05:14:14.320348+0000","last_unstale":"2024-09-11T05:28:13.436289+0000","last_undegraded":"2024-09-11T05:14:13.908501+0000","last_fullsized":"2024-09-11T05:14:13.761789+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19971,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935613+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:27:44.935613+0000","last_peered":"2024-09-11T05:27:44.935613+0000","last_clean":"2024-09-11T05:27:44.935613+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:27:44.935613+0000","last_undegraded":"2024-09-11T05:27:44.935613+0000","last_fullsized":"2024-09-11T05:27:44.935613+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936329+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:27:44.936329+0000","last_peered":"2024-09-11T05:27:44.936329+0000","last_clean":"2024-09-11T05:27:44.936329+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:27:44.936329+0000","last_undegraded":"2024-09-11T05:27:44.936329+0000","last_fullsized":"2024-09-11T05:27:44.936329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.935748+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:27:44.935748+0000","last_peered":"2024-09-11T05:27:44.935748+0000","last_clean":"2024-09-11T05:27:44.935748+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:27:44.935748+0000","last_undegraded":"2024-09-11T05:27:44.935748+0000","last_fullsized":"2024-09-11T05:27:44.935748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19776,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936203+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:27:44.936203+0000","last_peered":"2024-09-11T05:27:44.936203+0000","last_clean":"2024-09-11T05:27:44.936203+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:27:44.936203+0000","last_undegraded":"2024-09-11T05:27:44.936203+0000","last_fullsized":"2024-09-11T05:27:44.936203+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42234,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501480,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870360,"statfs":{"total":96636764160,"available":96123248640,"internally_reserved":0,"allocated":548864,"data_stored":183233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":512945835},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215292,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:19.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:19.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:19.569 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:19.586 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":548,"stamp":"2024-09-11T05:28:17.847402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1387004,"kb_used_data":71672,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281728516,"statfs":{"total":289910292480,"available":288490000384,"internally_reserved":0,"allocated":73392128,"data_stored":72291846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002363"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27671,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.939124+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:27:44.939124+0000","last_peered":"2024-09-11T05:27:44.939124+0000","last_clean":"2024-09-11T05:27:44.939124+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:27:44.939124+0000","last_undegraded":"2024-09-11T05:27:44.939124+0000","last_fullsized":"2024-09-11T05:27:44.939124+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26893,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567010+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:28:16.567010+0000","last_peered":"2024-09-11T05:28:16.567010+0000","last_clean":"2024-09-11T05:28:16.567010+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:16.567010+0000","last_undegraded":"2024-09-11T05:28:16.567010+0000","last_fullsized":"2024-09-11T05:28:16.567010+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20343,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938593+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:27:44.938593+0000","last_peered":"2024-09-11T05:27:44.938593+0000","last_clean":"2024-09-11T05:27:44.938593+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:27:44.938593+0000","last_undegraded":"2024-09-11T05:27:44.938593+0000","last_fullsized":"2024-09-11T05:27:44.938593+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27430,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566595+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:28:16.566595+0000","last_peered":"2024-09-11T05:28:16.566595+0000","last_clean":"2024-09-11T05:28:16.566595+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:16.566595+0000","last_undegraded":"2024-09-11T05:28:16.566595+0000","last_fullsized":"2024-09-11T05:28:16.566595+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566473+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:28:16.566473+0000","last_peered":"2024-09-11T05:28:16.566473+0000","last_clean":"2024-09-11T05:28:16.566473+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:16.566473+0000","last_undegraded":"2024-09-11T05:28:16.566473+0000","last_fullsized":"2024-09-11T05:28:16.566473+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20053,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938216+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:27:44.938216+0000","last_peered":"2024-09-11T05:27:44.938216+0000","last_clean":"2024-09-11T05:27:44.938216+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:27:44.938216+0000","last_undegraded":"2024-09-11T05:27:44.938216+0000","last_fullsized":"2024-09-11T05:27:44.938216+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566299+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:28:16.566299+0000","last_peered":"2024-09-11T05:28:16.566299+0000","last_clean":"2024-09-11T05:28:16.566299+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:16.566299+0000","last_undegraded":"2024-09-11T05:28:16.566299+0000","last_fullsized":"2024-09-11T05:28:16.566299+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566763+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:28:16.566763+0000","last_peered":"2024-09-11T05:28:16.566763+0000","last_clean":"2024-09-11T05:28:16.566763+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:16.566763+0000","last_undegraded":"2024-09-11T05:28:16.566763+0000","last_fullsized":"2024-09-11T05:28:16.566763+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938390+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:27:44.938390+0000","last_peered":"2024-09-11T05:27:44.938390+0000","last_clean":"2024-09-11T05:27:44.938390+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:27:44.938390+0000","last_undegraded":"2024-09-11T05:27:44.938390+0000","last_fullsized":"2024-09-11T05:27:44.938390+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.938392+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:27:44.938392+0000","last_peered":"2024-09-11T05:27:44.938392+0000","last_clean":"2024-09-11T05:27:44.938392+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:27:44.938392+0000","last_undegraded":"2024-09-11T05:27:44.938392+0000","last_fullsized":"2024-09-11T05:27:44.938392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566442+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:28:16.566442+0000","last_peered":"2024-09-11T05:28:16.566442+0000","last_clean":"2024-09-11T05:28:16.566442+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:16.566442+0000","last_undegraded":"2024-09-11T05:28:16.566442+0000","last_fullsized":"2024-09-11T05:28:16.566442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20300,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567388+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:28:16.567388+0000","last_peered":"2024-09-11T05:28:16.567388+0000","last_clean":"2024-09-11T05:28:16.567388+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:16.567388+0000","last_undegraded":"2024-09-11T05:28:16.567388+0000","last_fullsized":"2024-09-11T05:28:16.567388+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567074+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:28:16.567074+0000","last_peered":"2024-09-11T05:28:16.567074+0000","last_clean":"2024-09-11T05:28:16.567074+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:16.567074+0000","last_undegraded":"2024-09-11T05:28:16.567074+0000","last_fullsized":"2024-09-11T05:28:16.567074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.689521+0000","last_change":"2024-09-11T05:28:16.686309+0000","last_active":"2024-09-11T05:28:16.689521+0000","last_peered":"2024-09-11T05:28:16.689521+0000","last_clean":"2024-09-11T05:28:16.689521+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:16.689521+0000","last_undegraded":"2024-09-11T05:28:16.689521+0000","last_fullsized":"2024-09-11T05:28:16.689521+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:21.736022+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19976,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567223+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:28:16.567223+0000","last_peered":"2024-09-11T05:28:16.567223+0000","last_clean":"2024-09-11T05:28:16.567223+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:16.567223+0000","last_undegraded":"2024-09-11T05:28:16.567223+0000","last_fullsized":"2024-09-11T05:28:16.567223+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567850+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:28:16.567850+0000","last_peered":"2024-09-11T05:28:16.567850+0000","last_clean":"2024-09-11T05:28:16.567850+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:16.567850+0000","last_undegraded":"2024-09-11T05:28:16.567850+0000","last_fullsized":"2024-09-11T05:28:16.567850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567173+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:28:16.567173+0000","last_peered":"2024-09-11T05:28:16.567173+0000","last_clean":"2024-09-11T05:28:16.567173+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:16.567173+0000","last_undegraded":"2024-09-11T05:28:16.567173+0000","last_fullsized":"2024-09-11T05:28:16.567173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566828+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:28:16.566828+0000","last_peered":"2024-09-11T05:28:16.566828+0000","last_clean":"2024-09-11T05:28:16.566828+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:16.566828+0000","last_undegraded":"2024-09-11T05:28:16.566828+0000","last_fullsized":"2024-09-11T05:28:16.566828+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71745536,"data_stored":71738578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215292,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515836,"kb_used_data":56612,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93856004,"statfs":{"total":96636764160,"available":96108548096,"internally_reserved":0,"allocated":57970688,"data_stored":57605057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":57422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:19.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-11T05:28:19.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-11T05:28:19.588 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:19.887 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:19.888 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:19.905 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-09-11T05:28:19.847944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1385592,"kb_used_data":70264,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281729928,"statfs":{"total":289910292480,"available":288491446272,"internally_reserved":0,"allocated":71950336,"data_stored":70839109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002312"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27675,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571645+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:28:16.571645+0000","last_peered":"2024-09-11T05:28:16.571645+0000","last_clean":"2024-09-11T05:28:16.571645+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:28:16.571645+0000","last_undegraded":"2024-09-11T05:28:16.571645+0000","last_fullsized":"2024-09-11T05:28:16.571645+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26893,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567010+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:28:16.567010+0000","last_peered":"2024-09-11T05:28:16.567010+0000","last_clean":"2024-09-11T05:28:16.567010+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:16.567010+0000","last_undegraded":"2024-09-11T05:28:16.567010+0000","last_fullsized":"2024-09-11T05:28:16.567010+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20347,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:28:16.571129+0000","last_peered":"2024-09-11T05:28:16.571129+0000","last_clean":"2024-09-11T05:28:16.571129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:28:16.571129+0000","last_undegraded":"2024-09-11T05:28:16.571129+0000","last_fullsized":"2024-09-11T05:28:16.571129+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27430,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566595+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:28:16.566595+0000","last_peered":"2024-09-11T05:28:16.566595+0000","last_clean":"2024-09-11T05:28:16.566595+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:16.566595+0000","last_undegraded":"2024-09-11T05:28:16.566595+0000","last_fullsized":"2024-09-11T05:28:16.566595+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566473+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:28:16.566473+0000","last_peered":"2024-09-11T05:28:16.566473+0000","last_clean":"2024-09-11T05:28:16.566473+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:16.566473+0000","last_undegraded":"2024-09-11T05:28:16.566473+0000","last_fullsized":"2024-09-11T05:28:16.566473+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20057,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566299+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:28:16.566299+0000","last_peered":"2024-09-11T05:28:16.566299+0000","last_clean":"2024-09-11T05:28:16.566299+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:16.566299+0000","last_undegraded":"2024-09-11T05:28:16.566299+0000","last_fullsized":"2024-09-11T05:28:16.566299+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566763+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:28:16.566763+0000","last_peered":"2024-09-11T05:28:16.566763+0000","last_clean":"2024-09-11T05:28:16.566763+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:16.566763+0000","last_undegraded":"2024-09-11T05:28:16.566763+0000","last_fullsized":"2024-09-11T05:28:16.566763+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26782,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571036+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:28:16.571036+0000","last_peered":"2024-09-11T05:28:16.571036+0000","last_clean":"2024-09-11T05:28:16.571036+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:28:16.571036+0000","last_undegraded":"2024-09-11T05:28:16.571036+0000","last_fullsized":"2024-09-11T05:28:16.571036+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566442+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:28:16.566442+0000","last_peered":"2024-09-11T05:28:16.566442+0000","last_clean":"2024-09-11T05:28:16.566442+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:16.566442+0000","last_undegraded":"2024-09-11T05:28:16.566442+0000","last_fullsized":"2024-09-11T05:28:16.566442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20300,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567388+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:28:16.567388+0000","last_peered":"2024-09-11T05:28:16.567388+0000","last_clean":"2024-09-11T05:28:16.567388+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:16.567388+0000","last_undegraded":"2024-09-11T05:28:16.567388+0000","last_fullsized":"2024-09-11T05:28:16.567388+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567074+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:28:16.567074+0000","last_peered":"2024-09-11T05:28:16.567074+0000","last_clean":"2024-09-11T05:28:16.567074+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:16.567074+0000","last_undegraded":"2024-09-11T05:28:16.567074+0000","last_fullsized":"2024-09-11T05:28:16.567074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.689521+0000","last_change":"2024-09-11T05:28:16.686309+0000","last_active":"2024-09-11T05:28:16.689521+0000","last_peered":"2024-09-11T05:28:16.689521+0000","last_clean":"2024-09-11T05:28:16.689521+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:16.689521+0000","last_undegraded":"2024-09-11T05:28:16.689521+0000","last_fullsized":"2024-09-11T05:28:16.689521+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:21.736022+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19976,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567223+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:28:16.567223+0000","last_peered":"2024-09-11T05:28:16.567223+0000","last_clean":"2024-09-11T05:28:16.567223+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:16.567223+0000","last_undegraded":"2024-09-11T05:28:16.567223+0000","last_fullsized":"2024-09-11T05:28:16.567223+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567850+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:28:16.567850+0000","last_peered":"2024-09-11T05:28:16.567850+0000","last_clean":"2024-09-11T05:28:16.567850+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:16.567850+0000","last_undegraded":"2024-09-11T05:28:16.567850+0000","last_fullsized":"2024-09-11T05:28:16.567850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567173+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:28:16.567173+0000","last_peered":"2024-09-11T05:28:16.567173+0000","last_clean":"2024-09-11T05:28:16.567173+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:16.567173+0000","last_undegraded":"2024-09-11T05:28:16.567173+0000","last_fullsized":"2024-09-11T05:28:16.567173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566828+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:28:16.566828+0000","last_peered":"2024-09-11T05:28:16.566828+0000","last_clean":"2024-09-11T05:28:16.566828+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:16.566828+0000","last_undegraded":"2024-09-11T05:28:16.566828+0000","last_fullsized":"2024-09-11T05:28:16.566828+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70280402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514424,"kb_used_data":55204,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93857416,"statfs":{"total":96636764160,"available":96109993984,"internally_reserved":0,"allocated":56528896,"data_stored":56152320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":55967744,"data_stored":55964265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:19.906 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:20.192 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:20.193 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:20.208 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-09-11T05:28:19.847944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1385592,"kb_used_data":70264,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281729928,"statfs":{"total":289910292480,"available":288491446272,"internally_reserved":0,"allocated":71950336,"data_stored":70839109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002312"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27675,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571645+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:28:16.571645+0000","last_peered":"2024-09-11T05:28:16.571645+0000","last_clean":"2024-09-11T05:28:16.571645+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:28:16.571645+0000","last_undegraded":"2024-09-11T05:28:16.571645+0000","last_fullsized":"2024-09-11T05:28:16.571645+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26893,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567010+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:28:16.567010+0000","last_peered":"2024-09-11T05:28:16.567010+0000","last_clean":"2024-09-11T05:28:16.567010+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:16.567010+0000","last_undegraded":"2024-09-11T05:28:16.567010+0000","last_fullsized":"2024-09-11T05:28:16.567010+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20347,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:28:16.571129+0000","last_peered":"2024-09-11T05:28:16.571129+0000","last_clean":"2024-09-11T05:28:16.571129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:28:16.571129+0000","last_undegraded":"2024-09-11T05:28:16.571129+0000","last_fullsized":"2024-09-11T05:28:16.571129+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27430,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566595+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:28:16.566595+0000","last_peered":"2024-09-11T05:28:16.566595+0000","last_clean":"2024-09-11T05:28:16.566595+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:16.566595+0000","last_undegraded":"2024-09-11T05:28:16.566595+0000","last_fullsized":"2024-09-11T05:28:16.566595+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566473+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:28:16.566473+0000","last_peered":"2024-09-11T05:28:16.566473+0000","last_clean":"2024-09-11T05:28:16.566473+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:16.566473+0000","last_undegraded":"2024-09-11T05:28:16.566473+0000","last_fullsized":"2024-09-11T05:28:16.566473+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20057,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566299+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:28:16.566299+0000","last_peered":"2024-09-11T05:28:16.566299+0000","last_clean":"2024-09-11T05:28:16.566299+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:16.566299+0000","last_undegraded":"2024-09-11T05:28:16.566299+0000","last_fullsized":"2024-09-11T05:28:16.566299+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566763+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:28:16.566763+0000","last_peered":"2024-09-11T05:28:16.566763+0000","last_clean":"2024-09-11T05:28:16.566763+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:16.566763+0000","last_undegraded":"2024-09-11T05:28:16.566763+0000","last_fullsized":"2024-09-11T05:28:16.566763+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26782,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571036+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:28:16.571036+0000","last_peered":"2024-09-11T05:28:16.571036+0000","last_clean":"2024-09-11T05:28:16.571036+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:28:16.571036+0000","last_undegraded":"2024-09-11T05:28:16.571036+0000","last_fullsized":"2024-09-11T05:28:16.571036+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566442+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:28:16.566442+0000","last_peered":"2024-09-11T05:28:16.566442+0000","last_clean":"2024-09-11T05:28:16.566442+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:16.566442+0000","last_undegraded":"2024-09-11T05:28:16.566442+0000","last_fullsized":"2024-09-11T05:28:16.566442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20300,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567388+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:28:16.567388+0000","last_peered":"2024-09-11T05:28:16.567388+0000","last_clean":"2024-09-11T05:28:16.567388+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:16.567388+0000","last_undegraded":"2024-09-11T05:28:16.567388+0000","last_fullsized":"2024-09-11T05:28:16.567388+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567074+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:28:16.567074+0000","last_peered":"2024-09-11T05:28:16.567074+0000","last_clean":"2024-09-11T05:28:16.567074+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:16.567074+0000","last_undegraded":"2024-09-11T05:28:16.567074+0000","last_fullsized":"2024-09-11T05:28:16.567074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.689521+0000","last_change":"2024-09-11T05:28:16.686309+0000","last_active":"2024-09-11T05:28:16.689521+0000","last_peered":"2024-09-11T05:28:16.689521+0000","last_clean":"2024-09-11T05:28:16.689521+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:16.689521+0000","last_undegraded":"2024-09-11T05:28:16.689521+0000","last_fullsized":"2024-09-11T05:28:16.689521+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:21.736022+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19976,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567223+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:28:16.567223+0000","last_peered":"2024-09-11T05:28:16.567223+0000","last_clean":"2024-09-11T05:28:16.567223+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:16.567223+0000","last_undegraded":"2024-09-11T05:28:16.567223+0000","last_fullsized":"2024-09-11T05:28:16.567223+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567850+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:28:16.567850+0000","last_peered":"2024-09-11T05:28:16.567850+0000","last_clean":"2024-09-11T05:28:16.567850+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:16.567850+0000","last_undegraded":"2024-09-11T05:28:16.567850+0000","last_fullsized":"2024-09-11T05:28:16.567850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567173+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:28:16.567173+0000","last_peered":"2024-09-11T05:28:16.567173+0000","last_clean":"2024-09-11T05:28:16.567173+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:16.567173+0000","last_undegraded":"2024-09-11T05:28:16.567173+0000","last_fullsized":"2024-09-11T05:28:16.567173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566828+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:28:16.566828+0000","last_peered":"2024-09-11T05:28:16.566828+0000","last_clean":"2024-09-11T05:28:16.566828+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:16.566828+0000","last_undegraded":"2024-09-11T05:28:16.566828+0000","last_fullsized":"2024-09-11T05:28:16.566828+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70280402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514424,"kb_used_data":55204,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93857416,"statfs":{"total":96636764160,"available":96109993984,"internally_reserved":0,"allocated":56528896,"data_stored":56152320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":55967744,"data_stored":55964265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:20.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-11T05:28:20.210 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-11T05:28:20.218 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T05:28:20.218 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:20.504 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:20.505 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:20.522 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-09-11T05:28:19.847944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1385592,"kb_used_data":70264,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281729928,"statfs":{"total":289910292480,"available":288491446272,"internally_reserved":0,"allocated":71950336,"data_stored":70839109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002312"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27675,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571645+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:28:16.571645+0000","last_peered":"2024-09-11T05:28:16.571645+0000","last_clean":"2024-09-11T05:28:16.571645+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:28:16.571645+0000","last_undegraded":"2024-09-11T05:28:16.571645+0000","last_fullsized":"2024-09-11T05:28:16.571645+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26893,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567010+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:28:16.567010+0000","last_peered":"2024-09-11T05:28:16.567010+0000","last_clean":"2024-09-11T05:28:16.567010+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:16.567010+0000","last_undegraded":"2024-09-11T05:28:16.567010+0000","last_fullsized":"2024-09-11T05:28:16.567010+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20347,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:28:16.571129+0000","last_peered":"2024-09-11T05:28:16.571129+0000","last_clean":"2024-09-11T05:28:16.571129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:28:16.571129+0000","last_undegraded":"2024-09-11T05:28:16.571129+0000","last_fullsized":"2024-09-11T05:28:16.571129+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27430,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566595+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:28:16.566595+0000","last_peered":"2024-09-11T05:28:16.566595+0000","last_clean":"2024-09-11T05:28:16.566595+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:16.566595+0000","last_undegraded":"2024-09-11T05:28:16.566595+0000","last_fullsized":"2024-09-11T05:28:16.566595+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566473+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:28:16.566473+0000","last_peered":"2024-09-11T05:28:16.566473+0000","last_clean":"2024-09-11T05:28:16.566473+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:16.566473+0000","last_undegraded":"2024-09-11T05:28:16.566473+0000","last_fullsized":"2024-09-11T05:28:16.566473+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20057,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566299+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:28:16.566299+0000","last_peered":"2024-09-11T05:28:16.566299+0000","last_clean":"2024-09-11T05:28:16.566299+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:16.566299+0000","last_undegraded":"2024-09-11T05:28:16.566299+0000","last_fullsized":"2024-09-11T05:28:16.566299+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566763+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:28:16.566763+0000","last_peered":"2024-09-11T05:28:16.566763+0000","last_clean":"2024-09-11T05:28:16.566763+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:16.566763+0000","last_undegraded":"2024-09-11T05:28:16.566763+0000","last_fullsized":"2024-09-11T05:28:16.566763+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26782,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571036+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:28:16.571036+0000","last_peered":"2024-09-11T05:28:16.571036+0000","last_clean":"2024-09-11T05:28:16.571036+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:28:16.571036+0000","last_undegraded":"2024-09-11T05:28:16.571036+0000","last_fullsized":"2024-09-11T05:28:16.571036+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566442+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:28:16.566442+0000","last_peered":"2024-09-11T05:28:16.566442+0000","last_clean":"2024-09-11T05:28:16.566442+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:16.566442+0000","last_undegraded":"2024-09-11T05:28:16.566442+0000","last_fullsized":"2024-09-11T05:28:16.566442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20300,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567388+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:28:16.567388+0000","last_peered":"2024-09-11T05:28:16.567388+0000","last_clean":"2024-09-11T05:28:16.567388+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:16.567388+0000","last_undegraded":"2024-09-11T05:28:16.567388+0000","last_fullsized":"2024-09-11T05:28:16.567388+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567074+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:28:16.567074+0000","last_peered":"2024-09-11T05:28:16.567074+0000","last_clean":"2024-09-11T05:28:16.567074+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:16.567074+0000","last_undegraded":"2024-09-11T05:28:16.567074+0000","last_fullsized":"2024-09-11T05:28:16.567074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.689521+0000","last_change":"2024-09-11T05:28:16.686309+0000","last_active":"2024-09-11T05:28:16.689521+0000","last_peered":"2024-09-11T05:28:16.689521+0000","last_clean":"2024-09-11T05:28:16.689521+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:16.689521+0000","last_undegraded":"2024-09-11T05:28:16.689521+0000","last_fullsized":"2024-09-11T05:28:16.689521+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:21.736022+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19976,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567223+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:28:16.567223+0000","last_peered":"2024-09-11T05:28:16.567223+0000","last_clean":"2024-09-11T05:28:16.567223+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:16.567223+0000","last_undegraded":"2024-09-11T05:28:16.567223+0000","last_fullsized":"2024-09-11T05:28:16.567223+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567850+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:28:16.567850+0000","last_peered":"2024-09-11T05:28:16.567850+0000","last_clean":"2024-09-11T05:28:16.567850+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:16.567850+0000","last_undegraded":"2024-09-11T05:28:16.567850+0000","last_fullsized":"2024-09-11T05:28:16.567850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567173+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:28:16.567173+0000","last_peered":"2024-09-11T05:28:16.567173+0000","last_clean":"2024-09-11T05:28:16.567173+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:16.567173+0000","last_undegraded":"2024-09-11T05:28:16.567173+0000","last_fullsized":"2024-09-11T05:28:16.567173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566828+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:28:16.566828+0000","last_peered":"2024-09-11T05:28:16.566828+0000","last_clean":"2024-09-11T05:28:16.566828+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:16.566828+0000","last_undegraded":"2024-09-11T05:28:16.566828+0000","last_fullsized":"2024-09-11T05:28:16.566828+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70280402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514424,"kb_used_data":55204,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93857416,"statfs":{"total":96636764160,"available":96109993984,"internally_reserved":0,"allocated":56528896,"data_stored":56152320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":55967744,"data_stored":55964265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:20.523 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:20.816 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:20.834 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-09-11T05:28:19.847944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1385592,"kb_used_data":70264,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281729928,"statfs":{"total":289910292480,"available":288491446272,"internally_reserved":0,"allocated":71950336,"data_stored":70839109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002312"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27675,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571645+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:28:16.571645+0000","last_peered":"2024-09-11T05:28:16.571645+0000","last_clean":"2024-09-11T05:28:16.571645+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:28:16.571645+0000","last_undegraded":"2024-09-11T05:28:16.571645+0000","last_fullsized":"2024-09-11T05:28:16.571645+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26893,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567010+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:28:16.567010+0000","last_peered":"2024-09-11T05:28:16.567010+0000","last_clean":"2024-09-11T05:28:16.567010+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:16.567010+0000","last_undegraded":"2024-09-11T05:28:16.567010+0000","last_fullsized":"2024-09-11T05:28:16.567010+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20347,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:28:16.571129+0000","last_peered":"2024-09-11T05:28:16.571129+0000","last_clean":"2024-09-11T05:28:16.571129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:28:16.571129+0000","last_undegraded":"2024-09-11T05:28:16.571129+0000","last_fullsized":"2024-09-11T05:28:16.571129+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27430,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566595+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:28:16.566595+0000","last_peered":"2024-09-11T05:28:16.566595+0000","last_clean":"2024-09-11T05:28:16.566595+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:16.566595+0000","last_undegraded":"2024-09-11T05:28:16.566595+0000","last_fullsized":"2024-09-11T05:28:16.566595+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566473+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:28:16.566473+0000","last_peered":"2024-09-11T05:28:16.566473+0000","last_clean":"2024-09-11T05:28:16.566473+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:16.566473+0000","last_undegraded":"2024-09-11T05:28:16.566473+0000","last_fullsized":"2024-09-11T05:28:16.566473+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20057,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566299+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:28:16.566299+0000","last_peered":"2024-09-11T05:28:16.566299+0000","last_clean":"2024-09-11T05:28:16.566299+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:16.566299+0000","last_undegraded":"2024-09-11T05:28:16.566299+0000","last_fullsized":"2024-09-11T05:28:16.566299+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566763+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:28:16.566763+0000","last_peered":"2024-09-11T05:28:16.566763+0000","last_clean":"2024-09-11T05:28:16.566763+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:16.566763+0000","last_undegraded":"2024-09-11T05:28:16.566763+0000","last_fullsized":"2024-09-11T05:28:16.566763+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26782,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571036+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:28:16.571036+0000","last_peered":"2024-09-11T05:28:16.571036+0000","last_clean":"2024-09-11T05:28:16.571036+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:28:16.571036+0000","last_undegraded":"2024-09-11T05:28:16.571036+0000","last_fullsized":"2024-09-11T05:28:16.571036+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566442+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:28:16.566442+0000","last_peered":"2024-09-11T05:28:16.566442+0000","last_clean":"2024-09-11T05:28:16.566442+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:16.566442+0000","last_undegraded":"2024-09-11T05:28:16.566442+0000","last_fullsized":"2024-09-11T05:28:16.566442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20300,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567388+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:28:16.567388+0000","last_peered":"2024-09-11T05:28:16.567388+0000","last_clean":"2024-09-11T05:28:16.567388+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:16.567388+0000","last_undegraded":"2024-09-11T05:28:16.567388+0000","last_fullsized":"2024-09-11T05:28:16.567388+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567074+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:28:16.567074+0000","last_peered":"2024-09-11T05:28:16.567074+0000","last_clean":"2024-09-11T05:28:16.567074+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:16.567074+0000","last_undegraded":"2024-09-11T05:28:16.567074+0000","last_fullsized":"2024-09-11T05:28:16.567074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.689521+0000","last_change":"2024-09-11T05:28:16.686309+0000","last_active":"2024-09-11T05:28:16.689521+0000","last_peered":"2024-09-11T05:28:16.689521+0000","last_clean":"2024-09-11T05:28:16.689521+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:16.689521+0000","last_undegraded":"2024-09-11T05:28:16.689521+0000","last_fullsized":"2024-09-11T05:28:16.689521+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:21.736022+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19976,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567223+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:28:16.567223+0000","last_peered":"2024-09-11T05:28:16.567223+0000","last_clean":"2024-09-11T05:28:16.567223+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:16.567223+0000","last_undegraded":"2024-09-11T05:28:16.567223+0000","last_fullsized":"2024-09-11T05:28:16.567223+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567850+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:28:16.567850+0000","last_peered":"2024-09-11T05:28:16.567850+0000","last_clean":"2024-09-11T05:28:16.567850+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:16.567850+0000","last_undegraded":"2024-09-11T05:28:16.567850+0000","last_fullsized":"2024-09-11T05:28:16.567850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567173+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:28:16.567173+0000","last_peered":"2024-09-11T05:28:16.567173+0000","last_clean":"2024-09-11T05:28:16.567173+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:16.567173+0000","last_undegraded":"2024-09-11T05:28:16.567173+0000","last_fullsized":"2024-09-11T05:28:16.567173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566828+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:28:16.566828+0000","last_peered":"2024-09-11T05:28:16.566828+0000","last_clean":"2024-09-11T05:28:16.566828+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:16.566828+0000","last_undegraded":"2024-09-11T05:28:16.566828+0000","last_fullsized":"2024-09-11T05:28:16.566828+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70280402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514424,"kb_used_data":55204,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93857416,"statfs":{"total":96636764160,"available":96109993984,"internally_reserved":0,"allocated":56528896,"data_stored":56152320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":55967744,"data_stored":55964265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:20.835 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T05:28:20.835 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:21.121 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:21.122 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:21.136 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-09-11T05:28:19.847944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1385592,"kb_used_data":70264,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281729928,"statfs":{"total":289910292480,"available":288491446272,"internally_reserved":0,"allocated":71950336,"data_stored":70839109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002312"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27675,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571645+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:28:16.571645+0000","last_peered":"2024-09-11T05:28:16.571645+0000","last_clean":"2024-09-11T05:28:16.571645+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:28:16.571645+0000","last_undegraded":"2024-09-11T05:28:16.571645+0000","last_fullsized":"2024-09-11T05:28:16.571645+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20047,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936825+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:27:44.936825+0000","last_peered":"2024-09-11T05:27:44.936825+0000","last_clean":"2024-09-11T05:27:44.936825+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:27:44.936825+0000","last_undegraded":"2024-09-11T05:27:44.936825+0000","last_fullsized":"2024-09-11T05:27:44.936825+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26893,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567010+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:28:16.567010+0000","last_peered":"2024-09-11T05:28:16.567010+0000","last_clean":"2024-09-11T05:28:16.567010+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:16.567010+0000","last_undegraded":"2024-09-11T05:28:16.567010+0000","last_fullsized":"2024-09-11T05:28:16.567010+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20076,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936496+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:27:44.936496+0000","last_peered":"2024-09-11T05:27:44.936496+0000","last_clean":"2024-09-11T05:27:44.936496+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:27:44.936496+0000","last_undegraded":"2024-09-11T05:27:44.936496+0000","last_fullsized":"2024-09-11T05:27:44.936496+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20347,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:28:16.571129+0000","last_peered":"2024-09-11T05:28:16.571129+0000","last_clean":"2024-09-11T05:28:16.571129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:28:16.571129+0000","last_undegraded":"2024-09-11T05:28:16.571129+0000","last_fullsized":"2024-09-11T05:28:16.571129+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27430,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566595+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:28:16.566595+0000","last_peered":"2024-09-11T05:28:16.566595+0000","last_clean":"2024-09-11T05:28:16.566595+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:16.566595+0000","last_undegraded":"2024-09-11T05:28:16.566595+0000","last_fullsized":"2024-09-11T05:28:16.566595+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566473+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:28:16.566473+0000","last_peered":"2024-09-11T05:28:16.566473+0000","last_clean":"2024-09-11T05:28:16.566473+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:16.566473+0000","last_undegraded":"2024-09-11T05:28:16.566473+0000","last_fullsized":"2024-09-11T05:28:16.566473+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20057,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566299+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:28:16.566299+0000","last_peered":"2024-09-11T05:28:16.566299+0000","last_clean":"2024-09-11T05:28:16.566299+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:16.566299+0000","last_undegraded":"2024-09-11T05:28:16.566299+0000","last_fullsized":"2024-09-11T05:28:16.566299+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566763+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:28:16.566763+0000","last_peered":"2024-09-11T05:28:16.566763+0000","last_clean":"2024-09-11T05:28:16.566763+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:16.566763+0000","last_undegraded":"2024-09-11T05:28:16.566763+0000","last_fullsized":"2024-09-11T05:28:16.566763+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936261+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:27:44.936261+0000","last_peered":"2024-09-11T05:27:44.936261+0000","last_clean":"2024-09-11T05:27:44.936261+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:27:44.936261+0000","last_undegraded":"2024-09-11T05:27:44.936261+0000","last_fullsized":"2024-09-11T05:27:44.936261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26782,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571036+0000","last_change":"2024-09-11T05:27:12.608758+0000","last_active":"2024-09-11T05:28:16.571036+0000","last_peered":"2024-09-11T05:28:16.571036+0000","last_clean":"2024-09-11T05:28:16.571036+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:28:16.571036+0000","last_undegraded":"2024-09-11T05:28:16.571036+0000","last_fullsized":"2024-09-11T05:28:16.571036+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3300","last_scrub_stamp":"2024-09-11T05:14:09.784433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:09.784433+0000","objects_scrubbed":3239,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:19.753316+0000","scrub_duration":1595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936325+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:27:44.936325+0000","last_peered":"2024-09-11T05:27:44.936325+0000","last_clean":"2024-09-11T05:27:44.936325+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:27:44.936325+0000","last_undegraded":"2024-09-11T05:27:44.936325+0000","last_fullsized":"2024-09-11T05:27:44.936325+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566442+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:28:16.566442+0000","last_peered":"2024-09-11T05:28:16.566442+0000","last_clean":"2024-09-11T05:28:16.566442+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:16.566442+0000","last_undegraded":"2024-09-11T05:28:16.566442+0000","last_fullsized":"2024-09-11T05:28:16.566442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26839,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936046+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:27:44.936046+0000","last_peered":"2024-09-11T05:27:44.936046+0000","last_clean":"2024-09-11T05:27:44.936046+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:27:44.936046+0000","last_undegraded":"2024-09-11T05:27:44.936046+0000","last_fullsized":"2024-09-11T05:27:44.936046+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20300,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567388+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:28:16.567388+0000","last_peered":"2024-09-11T05:28:16.567388+0000","last_clean":"2024-09-11T05:28:16.567388+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:16.567388+0000","last_undegraded":"2024-09-11T05:28:16.567388+0000","last_fullsized":"2024-09-11T05:28:16.567388+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567074+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:28:16.567074+0000","last_peered":"2024-09-11T05:28:16.567074+0000","last_clean":"2024-09-11T05:28:16.567074+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:16.567074+0000","last_undegraded":"2024-09-11T05:28:16.567074+0000","last_fullsized":"2024-09-11T05:28:16.567074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.689521+0000","last_change":"2024-09-11T05:28:16.686309+0000","last_active":"2024-09-11T05:28:16.689521+0000","last_peered":"2024-09-11T05:28:16.689521+0000","last_clean":"2024-09-11T05:28:16.689521+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:16.689521+0000","last_undegraded":"2024-09-11T05:28:16.689521+0000","last_fullsized":"2024-09-11T05:28:16.689521+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:21.736022+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T05:27:44.936678+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:27:44.936678+0000","last_peered":"2024-09-11T05:27:44.936678+0000","last_clean":"2024-09-11T05:27:44.936678+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:27:44.936678+0000","last_undegraded":"2024-09-11T05:27:44.936678+0000","last_fullsized":"2024-09-11T05:27:44.936678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19976,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567223+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:28:16.567223+0000","last_peered":"2024-09-11T05:28:16.567223+0000","last_clean":"2024-09-11T05:28:16.567223+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:16.567223+0000","last_undegraded":"2024-09-11T05:28:16.567223+0000","last_fullsized":"2024-09-11T05:28:16.567223+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567850+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:28:16.567850+0000","last_peered":"2024-09-11T05:28:16.567850+0000","last_clean":"2024-09-11T05:28:16.567850+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:16.567850+0000","last_undegraded":"2024-09-11T05:28:16.567850+0000","last_fullsized":"2024-09-11T05:28:16.567850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567173+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:28:16.567173+0000","last_peered":"2024-09-11T05:28:16.567173+0000","last_clean":"2024-09-11T05:28:16.567173+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:16.567173+0000","last_undegraded":"2024-09-11T05:28:16.567173+0000","last_fullsized":"2024-09-11T05:28:16.567173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566828+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:28:16.566828+0000","last_peered":"2024-09-11T05:28:16.566828+0000","last_clean":"2024-09-11T05:28:16.566828+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:16.566828+0000","last_undegraded":"2024-09-11T05:28:16.566828+0000","last_fullsized":"2024-09-11T05:28:16.566828+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70287360,"data_stored":70280402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365504,"kb_used_data":14508,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":94006336,"statfs":{"total":96636764160,"available":96262488064,"internally_reserved":0,"allocated":14856192,"data_stored":14498117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359394987},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514424,"kb_used_data":55204,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93857416,"statfs":{"total":96636764160,"available":96109993984,"internally_reserved":0,"allocated":56528896,"data_stored":56152320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":55967744,"data_stored":55964265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14315520,"data_stored":14315520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:21.137 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T05:28:21.423 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:21.423 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":67,"fsid":"40251081-3433-4f55-b0e4-ce0789fd19fd","created":"2024-09-11T05:12:02.234737+0000","modified":"2024-09-11T05:28:16.549447+0000","last_up_change":"2024-09-11T05:14:11.716852+0000","last_in_change":"2024-09-11T05:14:03.662755+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T05:12:17.695239+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T05:12:29.395526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7b6888c-0033-4334-b02d-34bc88a2cbb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":24,"up_thru":65,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":3512043189}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":3512043189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":3512043189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":3512043189}]},"public_addr":"172.21.15.133:6804/3512043189","cluster_addr":"172.21.15.133:6805/3512043189","heartbeat_back_addr":"172.21.15.133:6807/3512043189","heartbeat_front_addr":"172.21.15.133:6806/3512043189","state":["exists","up"]},{"osd":1,"uuid":"0530ffc9-4ed5-4380-929c-e39eab601f4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":25,"up_from":41,"up_thru":66,"down_at":26,"lost_at":26,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":3859356444}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":3859356444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":3859356444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":3859356444}]},"public_addr":"172.21.15.133:6808/3859356444","cluster_addr":"172.21.15.133:6809/3859356444","heartbeat_back_addr":"172.21.15.133:6811/3859356444","heartbeat_front_addr":"172.21.15.133:6810/3859356444","state":["exists","up"]},{"osd":2,"uuid":"01d3910a-105e-4fa7-93fd-55d913165415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":55,"down_at":18,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":401696059}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":401696059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":401696059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":401696059}]},"public_addr":"172.21.15.133:6800/401696059","cluster_addr":"172.21.15.133:6801/401696059","heartbeat_back_addr":"172.21.15.133:6803/401696059","heartbeat_front_addr":"172.21.15.133:6802/401696059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:10.145955+0000","dead_epoch":21},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.764320+0000","dead_epoch":26},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T05:12:09.873913+0000","dead_epoch":18}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T05:28:22.441 INFO:tasks.ceph:Scrubbing osd.0 2024-09-11T05:28:22.442 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-11T05:28:22.600 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:28:22.601 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T05:28:22.601 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:28:22.616 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-11T05:28:22.902 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 0 to deep-scrub 2024-09-11T05:28:22.919 INFO:tasks.ceph:Scrubbing osd.1 2024-09-11T05:28:22.919 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-11T05:28:23.052 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:28:23.052 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T05:28:23.052 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:28:23.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-11T05:28:23.346 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 1 to deep-scrub 2024-09-11T05:28:23.364 INFO:tasks.ceph:Scrubbing osd.2 2024-09-11T05:28:23.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-11T05:28:23.496 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-11T05:28:23.497 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T05:28:23.497 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-11T05:28:23.511 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-11T05:28:23.780 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 2 to deep-scrub 2024-09-11T05:28:23.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:24.070 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:24.071 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:24.087 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":551,"stamp":"2024-09-11T05:28:23.849021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1352584,"kb_used_data":53900,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281762936,"statfs":{"total":289910292480,"available":288525246464,"internally_reserved":0,"allocated":55193600,"data_stored":54071428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1329777665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.299276"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27675,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571645+0000","last_change":"2024-09-11T05:26:42.615186+0000","last_active":"2024-09-11T05:28:16.571645+0000","last_peered":"2024-09-11T05:28:16.571645+0000","last_clean":"2024-09-11T05:28:16.571645+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:28:16.571645+0000","last_undegraded":"2024-09-11T05:28:16.571645+0000","last_fullsized":"2024-09-11T05:28:16.571645+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:19.588906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20051,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:17.202345+0000","last_change":"2024-09-11T05:13:45.037328+0000","last_active":"2024-09-11T05:28:17.202345+0000","last_peered":"2024-09-11T05:28:17.202345+0000","last_clean":"2024-09-11T05:28:17.202345+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:28:17.202345+0000","last_undegraded":"2024-09-11T05:28:17.202345+0000","last_fullsized":"2024-09-11T05:28:17.202345+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1638","last_scrub_stamp":"2024-09-11T05:13:45.035502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.035502+0000","objects_scrubbed":1596,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:26.442179+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26893,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567010+0000","last_change":"2024-09-11T05:22:17.822116+0000","last_active":"2024-09-11T05:28:16.567010+0000","last_peered":"2024-09-11T05:28:16.567010+0000","last_clean":"2024-09-11T05:28:16.567010+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:16.567010+0000","last_undegraded":"2024-09-11T05:28:16.567010+0000","last_fullsized":"2024-09-11T05:28:16.567010+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1632","last_scrub_stamp":"2024-09-11T05:13:45.985453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:45.985453+0000","objects_scrubbed":1593,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.942579+0000","scrub_duration":695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20080,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:17.202321+0000","last_change":"2024-09-11T05:13:22.498655+0000","last_active":"2024-09-11T05:28:17.202321+0000","last_peered":"2024-09-11T05:28:17.202321+0000","last_clean":"2024-09-11T05:28:17.202321+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:28:17.202321+0000","last_undegraded":"2024-09-11T05:28:17.202321+0000","last_fullsized":"2024-09-11T05:28:17.202321+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:12:30.129133+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:32.355648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20347,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.571129+0000","last_change":"2024-09-11T05:14:04.830016+0000","last_active":"2024-09-11T05:28:16.571129+0000","last_peered":"2024-09-11T05:28:16.571129+0000","last_clean":"2024-09-11T05:28:16.571129+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:28:16.571129+0000","last_undegraded":"2024-09-11T05:28:16.571129+0000","last_fullsized":"2024-09-11T05:28:16.571129+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3166","last_scrub_stamp":"2024-09-11T05:14:04.829816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:04.829816+0000","objects_scrubbed":3159,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:20.037849+0000","scrub_duration":583,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27430,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566595+0000","last_change":"2024-09-11T05:19:25.748675+0000","last_active":"2024-09-11T05:28:16.566595+0000","last_peered":"2024-09-11T05:28:16.566595+0000","last_clean":"2024-09-11T05:28:16.566595+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:16.566595+0000","last_undegraded":"2024-09-11T05:28:16.566595+0000","last_fullsized":"2024-09-11T05:28:16.566595+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3536","last_scrub_stamp":"2024-09-11T05:14:11.134821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:11.134821+0000","objects_scrubbed":3451,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:14.629621+0000","scrub_duration":1947,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22681,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566473+0000","last_change":"2024-09-11T05:23:55.990222+0000","last_active":"2024-09-11T05:28:16.566473+0000","last_peered":"2024-09-11T05:28:16.566473+0000","last_clean":"2024-09-11T05:28:16.566473+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:16.566473+0000","last_undegraded":"2024-09-11T05:28:16.566473+0000","last_fullsized":"2024-09-11T05:28:16.566473+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2868","last_scrub_stamp":"2024-09-11T05:14:00.287571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:00.287571+0000","objects_scrubbed":2829,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:15.458367+0000","scrub_duration":1083,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20057,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:13:59.199579+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2733","last_scrub_stamp":"2024-09-11T05:13:59.199391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:59.199391+0000","objects_scrubbed":2695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.928589+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566299+0000","last_change":"2024-09-11T05:14:13.773901+0000","last_active":"2024-09-11T05:28:16.566299+0000","last_peered":"2024-09-11T05:28:16.566299+0000","last_clean":"2024-09-11T05:28:16.566299+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:16.566299+0000","last_undegraded":"2024-09-11T05:28:16.566299+0000","last_fullsized":"2024-09-11T05:28:16.566299+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:43.569627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20145,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566763+0000","last_change":"2024-09-11T05:25:41.466593+0000","last_active":"2024-09-11T05:28:16.566763+0000","last_peered":"2024-09-11T05:28:16.566763+0000","last_clean":"2024-09-11T05:28:16.566763+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:16.566763+0000","last_undegraded":"2024-09-11T05:28:16.566763+0000","last_fullsized":"2024-09-11T05:28:16.566763+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1981","last_scrub_stamp":"2024-09-11T05:13:49.372780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.372780+0000","objects_scrubbed":1925,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:50.634701+0000","scrub_duration":1099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:17.201879+0000","last_change":"2024-09-11T05:13:22.497074+0000","last_active":"2024-09-11T05:28:17.201879+0000","last_peered":"2024-09-11T05:28:17.201879+0000","last_clean":"2024-09-11T05:28:17.201879+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:28:17.201879+0000","last_undegraded":"2024-09-11T05:28:17.201879+0000","last_fullsized":"2024-09-11T05:28:17.201879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:31:27.679881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26790,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:23.193810+0000","last_change":"2024-09-11T05:28:23.193810+0000","last_active":"2024-09-11T05:28:23.193810+0000","last_peered":"2024-09-11T05:28:23.193810+0000","last_clean":"2024-09-11T05:28:23.193810+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:28:23.193810+0000","last_undegraded":"2024-09-11T05:28:23.193810+0000","last_fullsized":"2024-09-11T05:28:23.193810+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19672","last_scrub_stamp":"2024-09-11T05:28:23.193726+0000","last_deep_scrub":"49'19672","last_deep_scrub_stamp":"2024-09-11T05:28:23.193726+0000","last_clean_scrub_stamp":"2024-09-11T05:28:23.193726+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:20.528859+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.570833+0000","last_change":"2024-09-11T05:14:13.904389+0000","last_active":"2024-09-11T05:28:16.570833+0000","last_peered":"2024-09-11T05:28:16.570833+0000","last_clean":"2024-09-11T05:28:16.570833+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:28:16.570833+0000","last_undegraded":"2024-09-11T05:28:16.570833+0000","last_fullsized":"2024-09-11T05:28:16.570833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:17.943939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27256,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:17.201894+0000","last_change":"2024-09-11T05:24:28.358586+0000","last_active":"2024-09-11T05:28:17.201894+0000","last_peered":"2024-09-11T05:28:17.201894+0000","last_clean":"2024-09-11T05:28:17.201894+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:28:17.201894+0000","last_undegraded":"2024-09-11T05:28:17.201894+0000","last_fullsized":"2024-09-11T05:28:17.201894+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1993","last_scrub_stamp":"2024-09-11T05:13:49.998266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:49.998266+0000","objects_scrubbed":1966,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:34.872806+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566442+0000","last_change":"2024-09-11T05:14:13.830732+0000","last_active":"2024-09-11T05:28:16.566442+0000","last_peered":"2024-09-11T05:28:16.566442+0000","last_clean":"2024-09-11T05:28:16.566442+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:16.566442+0000","last_undegraded":"2024-09-11T05:28:16.566442+0000","last_fullsized":"2024-09-11T05:28:16.566442+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:18.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26843,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:17.201831+0000","last_change":"2024-09-11T05:15:57.589970+0000","last_active":"2024-09-11T05:28:17.201831+0000","last_peered":"2024-09-11T05:28:17.201831+0000","last_clean":"2024-09-11T05:28:17.201831+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:28:17.201831+0000","last_undegraded":"2024-09-11T05:28:17.201831+0000","last_fullsized":"2024-09-11T05:28:17.201831+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2129","last_scrub_stamp":"2024-09-11T05:13:51.225394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:13:51.225394+0000","objects_scrubbed":2091,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:12.033668+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20300,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567388+0000","last_change":"2024-09-11T05:27:45.067933+0000","last_active":"2024-09-11T05:28:16.567388+0000","last_peered":"2024-09-11T05:28:16.567388+0000","last_clean":"2024-09-11T05:28:16.567388+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:16.567388+0000","last_undegraded":"2024-09-11T05:28:16.567388+0000","last_fullsized":"2024-09-11T05:28:16.567388+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3023","last_scrub_stamp":"2024-09-11T05:14:01.854947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:01.854947+0000","objects_scrubbed":2965,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:11.067180+0000","scrub_duration":1631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567074+0000","last_change":"2024-09-11T05:14:13.980857+0000","last_active":"2024-09-11T05:28:16.567074+0000","last_peered":"2024-09-11T05:28:16.567074+0000","last_clean":"2024-09-11T05:28:16.567074+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:16.567074+0000","last_undegraded":"2024-09-11T05:28:16.567074+0000","last_fullsized":"2024-09-11T05:28:16.567074+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:56.662423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19811,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.689521+0000","last_change":"2024-09-11T05:28:16.686309+0000","last_active":"2024-09-11T05:28:16.689521+0000","last_peered":"2024-09-11T05:28:16.689521+0000","last_clean":"2024-09-11T05:28:16.689521+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:16.689521+0000","last_undegraded":"2024-09-11T05:28:16.689521+0000","last_fullsized":"2024-09-11T05:28:16.689521+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3176","last_scrub_stamp":"2024-09-11T05:14:07.390153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:07.390153+0000","objects_scrubbed":3126,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:21.736022+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:17.202200+0000","last_change":"2024-09-11T05:13:22.497785+0000","last_active":"2024-09-11T05:28:17.202200+0000","last_peered":"2024-09-11T05:28:17.202200+0000","last_clean":"2024-09-11T05:28:17.202200+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:28:17.202200+0000","last_undegraded":"2024-09-11T05:28:17.202200+0000","last_fullsized":"2024-09-11T05:28:17.202200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:18.725527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19976,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567223+0000","last_change":"2024-09-11T05:26:12.065797+0000","last_active":"2024-09-11T05:28:16.567223+0000","last_peered":"2024-09-11T05:28:16.567223+0000","last_clean":"2024-09-11T05:28:16.567223+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:16.567223+0000","last_undegraded":"2024-09-11T05:28:16.567223+0000","last_fullsized":"2024-09-11T05:28:16.567223+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3288","last_scrub_stamp":"2024-09-11T05:14:08.586771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:08.586771+0000","objects_scrubbed":3238,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:15:22.052535+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567850+0000","last_change":"2024-09-11T05:14:13.983229+0000","last_active":"2024-09-11T05:28:16.567850+0000","last_peered":"2024-09-11T05:28:16.567850+0000","last_clean":"2024-09-11T05:28:16.567850+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:16.567850+0000","last_undegraded":"2024-09-11T05:28:16.567850+0000","last_fullsized":"2024-09-11T05:28:16.567850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.029697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.567173+0000","last_change":"2024-09-11T05:14:13.750447+0000","last_active":"2024-09-11T05:28:16.567173+0000","last_peered":"2024-09-11T05:28:16.567173+0000","last_clean":"2024-09-11T05:28:16.567173+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:16.567173+0000","last_undegraded":"2024-09-11T05:28:16.567173+0000","last_fullsized":"2024-09-11T05:28:16.567173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:18.438492+0000","last_clean_scrub_stamp":"2024-09-11T05:12:18.438492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:38.259217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19590","reported_seq":19780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:16.566828+0000","last_change":"2024-09-11T05:25:10.044266+0000","last_active":"2024-09-11T05:28:16.566828+0000","last_peered":"2024-09-11T05:28:16.566828+0000","last_clean":"2024-09-11T05:28:16.566828+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:16.566828+0000","last_undegraded":"2024-09-11T05:28:16.566828+0000","last_fullsized":"2024-09-11T05:28:16.566828+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3031","last_scrub_stamp":"2024-09-11T05:14:02.428529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:12:30.129133+0000","last_clean_scrub_stamp":"2024-09-11T05:14:02.428529+0000","objects_scrubbed":2982,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:07.798210+0000","scrub_duration":1183,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53514240,"data_stored":53507282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160930,"ondisk_log_size":160930,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986361,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334900,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":94036940,"statfs":{"total":96636764160,"available":96293826560,"internally_reserved":0,"allocated":557056,"data_stored":188036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659306,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505664,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866176,"statfs":{"total":96636764160,"available":96118964224,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512020,"kb_used_data":52804,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93859820,"statfs":{"total":96636764160,"available":96112455680,"internally_reserved":0,"allocated":54071296,"data_stored":53694720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53510144,"data_stored":53506665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:24.091 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-11T05:12:30.129133+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=30, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.091 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-11T05:13:45.035502+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=13, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.091 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-11T05:13:45.985453+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=13, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.091 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-11T05:12:30.129133+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=30, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-11T05:14:04.829816+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=14, tm_sec=4, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-11T05:14:11.134821+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=14, tm_sec=11, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-11T05:14:00.287571+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=14, tm_sec=0, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-11T05:13:59.199391+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=13, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-11T05:13:49.372780+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=13, tm_sec=49, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.092 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-11T05:13:49.998266+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=13, tm_sec=49, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.093 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.093 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-11T05:13:51.225394+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=13, tm_sec=51, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.093 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-11T05:14:01.854947+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=14, tm_sec=1, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.093 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.093 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-11T05:14:07.390153+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=14, tm_sec=7, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.094 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.094 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-11T05:14:08.586771+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=14, tm_sec=8, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.094 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.094 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-11T05:12:18.438492+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=12, tm_sec=18, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.094 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-11T05:14:02.428529+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=14, tm_sec=2, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=5, tm_min=28, tm_sec=21, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T05:28:24.094 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-11T05:28:44.096 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T05:28:44.406 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-11T05:28:44.406 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-11T05:28:44.424 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":561,"stamp":"2024-09-11T05:28:43.854396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281696,"num_write_kb":489679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42513,"num_bytes_recovered":30318611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1342972,"kb_used_data":44300,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281772548,"statfs":{"total":289910292480,"available":288535089152,"internally_reserved":0,"allocated":45363200,"data_stored":44241028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1329777665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003435"},"pg_stats":[{"pgid":"2.f","version":"49'20190","reported_seq":27683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:27.114025+0000","last_change":"2024-09-11T05:28:27.114025+0000","last_active":"2024-09-11T05:28:27.114025+0000","last_peered":"2024-09-11T05:28:27.114025+0000","last_clean":"2024-09-11T05:28:27.114025+0000","last_became_active":"2024-09-11T05:26:42.532148+0000","last_became_peered":"2024-09-11T05:26:42.532148+0000","last_unstale":"2024-09-11T05:28:27.114025+0000","last_undegraded":"2024-09-11T05:28:27.114025+0000","last_fullsized":"2024-09-11T05:28:27.114025+0000","mapping_epoch":60,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20190","last_scrub_stamp":"2024-09-11T05:28:27.113850+0000","last_deep_scrub":"49'20190","last_deep_scrub_stamp":"2024-09-11T05:28:27.113850+0000","last_clean_scrub_stamp":"2024-09-11T05:28:27.113850+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:22.639966+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20190,"num_write_kb":40380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3675,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19900","reported_seq":20059,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:27.299469+0000","last_change":"2024-09-11T05:28:27.299469+0000","last_active":"2024-09-11T05:28:27.299469+0000","last_peered":"2024-09-11T05:28:27.299469+0000","last_clean":"2024-09-11T05:28:27.299469+0000","last_became_active":"2024-09-11T05:13:22.498075+0000","last_became_peered":"2024-09-11T05:13:22.498075+0000","last_unstale":"2024-09-11T05:28:27.299469+0000","last_undegraded":"2024-09-11T05:28:27.299469+0000","last_fullsized":"2024-09-11T05:28:27.299469+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19900","last_scrub_stamp":"2024-09-11T05:28:27.299281+0000","last_deep_scrub":"49'19900","last_deep_scrub_stamp":"2024-09-11T05:28:27.299281+0000","last_clean_scrub_stamp":"2024-09-11T05:28:27.299281+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:32.221418+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18298,"num_write_kb":33404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19718","reported_seq":26901,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:35.059813+0000","last_change":"2024-09-11T05:28:35.059813+0000","last_active":"2024-09-11T05:28:35.059813+0000","last_peered":"2024-09-11T05:28:35.059813+0000","last_clean":"2024-09-11T05:28:35.059813+0000","last_became_active":"2024-09-11T05:22:17.755541+0000","last_became_peered":"2024-09-11T05:22:17.755541+0000","last_unstale":"2024-09-11T05:28:35.059813+0000","last_undegraded":"2024-09-11T05:28:35.059813+0000","last_fullsized":"2024-09-11T05:28:35.059813+0000","mapping_epoch":48,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19718","last_scrub_stamp":"2024-09-11T05:28:35.059613+0000","last_deep_scrub":"49'19718","last_deep_scrub_stamp":"2024-09-11T05:28:35.059613+0000","last_clean_scrub_stamp":"2024-09-11T05:28:35.059613+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:25.928238+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18121,"num_write_kb":33053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3495,"num_bytes_recovered":1323008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19960","reported_seq":20088,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:28.302499+0000","last_change":"2024-09-11T05:28:28.302499+0000","last_active":"2024-09-11T05:28:28.302499+0000","last_peered":"2024-09-11T05:28:28.302499+0000","last_clean":"2024-09-11T05:28:28.302499+0000","last_became_active":"2024-09-11T05:13:22.498383+0000","last_became_peered":"2024-09-11T05:13:22.498383+0000","last_unstale":"2024-09-11T05:28:28.302499+0000","last_undegraded":"2024-09-11T05:28:28.302499+0000","last_fullsized":"2024-09-11T05:28:28.302499+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19960","last_scrub_stamp":"2024-09-11T05:28:28.302281+0000","last_deep_scrub":"49'19960","last_deep_scrub_stamp":"2024-09-11T05:28:28.302281+0000","last_clean_scrub_stamp":"2024-09-11T05:28:28.302281+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:30.872870+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":39920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20169","reported_seq":20356,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:26.150329+0000","last_change":"2024-09-11T05:28:26.150096+0000","last_active":"2024-09-11T05:28:26.150329+0000","last_peered":"2024-09-11T05:28:26.150329+0000","last_clean":"2024-09-11T05:28:26.150329+0000","last_became_active":"2024-09-11T05:13:21.516484+0000","last_became_peered":"2024-09-11T05:13:21.516484+0000","last_unstale":"2024-09-11T05:28:26.150329+0000","last_undegraded":"2024-09-11T05:28:26.150329+0000","last_fullsized":"2024-09-11T05:28:26.150329+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20169","last_scrub_stamp":"2024-09-11T05:28:26.149884+0000","last_deep_scrub":"49'20169","last_deep_scrub_stamp":"2024-09-11T05:28:26.149884+0000","last_clean_scrub_stamp":"2024-09-11T05:28:26.149884+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:36:11.222504+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":27688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20035","reported_seq":27438,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:29.125569+0000","last_change":"2024-09-11T05:28:29.125569+0000","last_active":"2024-09-11T05:28:29.125569+0000","last_peered":"2024-09-11T05:28:29.125569+0000","last_clean":"2024-09-11T05:28:29.125569+0000","last_became_active":"2024-09-11T05:19:25.714851+0000","last_became_peered":"2024-09-11T05:19:25.714851+0000","last_unstale":"2024-09-11T05:28:29.125569+0000","last_undegraded":"2024-09-11T05:28:29.125569+0000","last_fullsized":"2024-09-11T05:28:29.125569+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20035","last_scrub_stamp":"2024-09-11T05:28:29.125385+0000","last_deep_scrub":"49'20035","last_deep_scrub_stamp":"2024-09-11T05:28:29.125385+0000","last_clean_scrub_stamp":"2024-09-11T05:28:29.125385+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:38:54.925277+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16579,"num_write_kb":26256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":14745600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20202","reported_seq":22689,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:34.093086+0000","last_change":"2024-09-11T05:28:34.093086+0000","last_active":"2024-09-11T05:28:34.093086+0000","last_peered":"2024-09-11T05:28:34.093086+0000","last_clean":"2024-09-11T05:28:34.093086+0000","last_became_active":"2024-09-11T05:23:55.898107+0000","last_became_peered":"2024-09-11T05:23:55.898107+0000","last_unstale":"2024-09-11T05:28:34.093086+0000","last_undegraded":"2024-09-11T05:28:34.093086+0000","last_fullsized":"2024-09-11T05:28:34.093086+0000","mapping_epoch":50,"log_start":"45'10200","ondisk_log_start":"45'10200","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20202","last_scrub_stamp":"2024-09-11T05:28:34.092902+0000","last_deep_scrub":"49'20202","last_deep_scrub_stamp":"2024-09-11T05:28:34.092902+0000","last_clean_scrub_stamp":"2024-09-11T05:28:34.092902+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:45.063273+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'19884","reported_seq":20065,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:25.174205+0000","last_change":"2024-09-11T05:28:25.174205+0000","last_active":"2024-09-11T05:28:25.174205+0000","last_peered":"2024-09-11T05:28:25.174205+0000","last_clean":"2024-09-11T05:28:25.174205+0000","last_became_active":"2024-09-11T05:13:21.507368+0000","last_became_peered":"2024-09-11T05:13:21.507368+0000","last_unstale":"2024-09-11T05:28:25.174205+0000","last_undegraded":"2024-09-11T05:28:25.174205+0000","last_fullsized":"2024-09-11T05:28:25.174205+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19884","last_scrub_stamp":"2024-09-11T05:28:25.174022+0000","last_deep_scrub":"49'19884","last_deep_scrub_stamp":"2024-09-11T05:28:25.174022+0000","last_clean_scrub_stamp":"2024-09-11T05:28:25.174022+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:36.534786+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17187,"num_write_kb":28984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:25.126732+0000","last_change":"2024-09-11T05:28:25.126732+0000","last_active":"2024-09-11T05:28:25.126732+0000","last_peered":"2024-09-11T05:28:25.126732+0000","last_clean":"2024-09-11T05:28:25.126732+0000","last_became_active":"2024-09-11T05:14:13.773081+0000","last_became_peered":"2024-09-11T05:14:13.773081+0000","last_unstale":"2024-09-11T05:28:25.126732+0000","last_undegraded":"2024-09-11T05:28:25.126732+0000","last_fullsized":"2024-09-11T05:28:25.126732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:28:25.126530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:28:25.126530+0000","last_clean_scrub_stamp":"2024-09-11T05:28:25.126530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:05:58.771023+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010938289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19971","reported_seq":20153,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:33.123482+0000","last_change":"2024-09-11T05:28:33.123482+0000","last_active":"2024-09-11T05:28:33.123482+0000","last_peered":"2024-09-11T05:28:33.123482+0000","last_clean":"2024-09-11T05:28:33.123482+0000","last_became_active":"2024-09-11T05:25:41.376086+0000","last_became_peered":"2024-09-11T05:25:41.376086+0000","last_unstale":"2024-09-11T05:28:33.123482+0000","last_undegraded":"2024-09-11T05:28:33.123482+0000","last_fullsized":"2024-09-11T05:28:33.123482+0000","mapping_epoch":56,"log_start":"45'9900","ondisk_log_start":"45'9900","created":15,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19971","last_scrub_stamp":"2024-09-11T05:28:33.123294+0000","last_deep_scrub":"49'19971","last_deep_scrub_stamp":"2024-09-11T05:28:33.123294+0000","last_clean_scrub_stamp":"2024-09-11T05:28:33.123294+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:58.930026+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18039,"num_write_kb":32228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:24.232738+0000","last_change":"2024-09-11T05:28:24.232738+0000","last_active":"2024-09-11T05:28:24.232738+0000","last_peered":"2024-09-11T05:28:24.232738+0000","last_clean":"2024-09-11T05:28:24.232738+0000","last_became_active":"2024-09-11T05:13:22.496787+0000","last_became_peered":"2024-09-11T05:13:22.496787+0000","last_unstale":"2024-09-11T05:28:24.232738+0000","last_undegraded":"2024-09-11T05:28:24.232738+0000","last_fullsized":"2024-09-11T05:28:24.232738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:28:24.231876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:28:24.231876+0000","last_clean_scrub_stamp":"2024-09-11T05:28:24.231876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:39.219805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19672","reported_seq":26791,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:23.193869+0000","last_change":"2024-09-11T05:28:23.193810+0000","last_active":"2024-09-11T05:28:23.193869+0000","last_peered":"2024-09-11T05:28:23.193869+0000","last_clean":"2024-09-11T05:28:23.193869+0000","last_became_active":"2024-09-11T05:27:12.548618+0000","last_became_peered":"2024-09-11T05:27:12.548618+0000","last_unstale":"2024-09-11T05:28:23.193869+0000","last_undegraded":"2024-09-11T05:28:23.193869+0000","last_fullsized":"2024-09-11T05:28:23.193869+0000","mapping_epoch":62,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19672","last_scrub_stamp":"2024-09-11T05:28:23.193726+0000","last_deep_scrub":"49'19672","last_deep_scrub_stamp":"2024-09-11T05:28:23.193726+0000","last_clean_scrub_stamp":"2024-09-11T05:28:23.193726+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:20.528859+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3444,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"67'3","reported_seq":164,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:24.190746+0000","last_change":"2024-09-11T05:28:24.190746+0000","last_active":"2024-09-11T05:28:24.190746+0000","last_peered":"2024-09-11T05:28:24.190746+0000","last_clean":"2024-09-11T05:28:24.190746+0000","last_became_active":"2024-09-11T05:14:13.904006+0000","last_became_peered":"2024-09-11T05:14:13.904006+0000","last_unstale":"2024-09-11T05:28:24.190746+0000","last_undegraded":"2024-09-11T05:28:24.190746+0000","last_fullsized":"2024-09-11T05:28:24.190746+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'3","last_scrub_stamp":"2024-09-11T05:28:24.189988+0000","last_deep_scrub":"67'3","last_deep_scrub_stamp":"2024-09-11T05:28:24.189988+0000","last_clean_scrub_stamp":"2024-09-11T05:28:24.189988+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:13:03.511291+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001247845,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20033","reported_seq":27264,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:25.244018+0000","last_change":"2024-09-11T05:28:25.244018+0000","last_active":"2024-09-11T05:28:25.244018+0000","last_peered":"2024-09-11T05:28:25.244018+0000","last_clean":"2024-09-11T05:28:25.244018+0000","last_became_active":"2024-09-11T05:24:28.297129+0000","last_became_peered":"2024-09-11T05:24:28.297129+0000","last_unstale":"2024-09-11T05:28:25.244018+0000","last_undegraded":"2024-09-11T05:28:25.244018+0000","last_fullsized":"2024-09-11T05:28:25.244018+0000","mapping_epoch":52,"log_start":"45'10000","ondisk_log_start":"45'10000","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20033","last_scrub_stamp":"2024-09-11T05:28:25.243844+0000","last_deep_scrub":"49'20033","last_deep_scrub_stamp":"2024-09-11T05:28:25.243844+0000","last_clean_scrub_stamp":"2024-09-11T05:28:25.243844+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:50.122057+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18062,"num_write_kb":32192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"67'2","reported_seq":137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:27.103066+0000","last_change":"2024-09-11T05:28:27.103066+0000","last_active":"2024-09-11T05:28:27.103066+0000","last_peered":"2024-09-11T05:28:27.103066+0000","last_clean":"2024-09-11T05:28:27.103066+0000","last_became_active":"2024-09-11T05:14:13.830295+0000","last_became_peered":"2024-09-11T05:14:13.830295+0000","last_unstale":"2024-09-11T05:28:27.103066+0000","last_undegraded":"2024-09-11T05:28:27.103066+0000","last_fullsized":"2024-09-11T05:28:27.103066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'2","last_scrub_stamp":"2024-09-11T05:28:27.102808+0000","last_deep_scrub":"67'2","last_deep_scrub_stamp":"2024-09-11T05:28:27.102808+0000","last_clean_scrub_stamp":"2024-09-11T05:28:27.102808+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:26.440699+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0013135009999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19708","reported_seq":26851,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:29.327166+0000","last_change":"2024-09-11T05:28:29.327166+0000","last_active":"2024-09-11T05:28:29.327166+0000","last_peered":"2024-09-11T05:28:29.327166+0000","last_clean":"2024-09-11T05:28:29.327166+0000","last_became_active":"2024-09-11T05:15:57.572636+0000","last_became_peered":"2024-09-11T05:15:57.572636+0000","last_unstale":"2024-09-11T05:28:29.327166+0000","last_undegraded":"2024-09-11T05:28:29.327166+0000","last_fullsized":"2024-09-11T05:28:29.327166+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19708","last_scrub_stamp":"2024-09-11T05:28:29.326988+0000","last_deep_scrub":"49'19708","last_deep_scrub_stamp":"2024-09-11T05:28:29.326988+0000","last_clean_scrub_stamp":"2024-09-11T05:28:29.326988+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:46.790246+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17611,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20144","reported_seq":20308,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:36.076316+0000","last_change":"2024-09-11T05:28:36.076316+0000","last_active":"2024-09-11T05:28:36.076316+0000","last_peered":"2024-09-11T05:28:36.076316+0000","last_clean":"2024-09-11T05:28:36.076316+0000","last_became_active":"2024-09-11T05:27:44.939715+0000","last_became_peered":"2024-09-11T05:27:44.939715+0000","last_unstale":"2024-09-11T05:28:36.076316+0000","last_undegraded":"2024-09-11T05:28:36.076316+0000","last_fullsized":"2024-09-11T05:28:36.076316+0000","mapping_epoch":64,"log_start":"45'10100","ondisk_log_start":"45'10100","created":15,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20144","last_scrub_stamp":"2024-09-11T05:28:36.076089+0000","last_deep_scrub":"49'20144","last_deep_scrub_stamp":"2024-09-11T05:28:36.076089+0000","last_clean_scrub_stamp":"2024-09-11T05:28:36.076089+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:11.337824+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17177,"num_write_kb":28424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:26.125703+0000","last_change":"2024-09-11T05:28:26.125703+0000","last_active":"2024-09-11T05:28:26.125703+0000","last_peered":"2024-09-11T05:28:26.125703+0000","last_clean":"2024-09-11T05:28:26.125703+0000","last_became_active":"2024-09-11T05:14:13.980498+0000","last_became_peered":"2024-09-11T05:14:13.980498+0000","last_unstale":"2024-09-11T05:28:26.125703+0000","last_undegraded":"2024-09-11T05:28:26.125703+0000","last_fullsized":"2024-09-11T05:28:26.125703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:28:26.125503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:28:26.125503+0000","last_clean_scrub_stamp":"2024-09-11T05:28:26.125503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:36.361201+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0021646479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19654","reported_seq":19819,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:30.064774+0000","last_change":"2024-09-11T05:28:30.064774+0000","last_active":"2024-09-11T05:28:30.064774+0000","last_peered":"2024-09-11T05:28:30.064774+0000","last_clean":"2024-09-11T05:28:30.064774+0000","last_became_active":"2024-09-11T05:28:16.572794+0000","last_became_peered":"2024-09-11T05:28:16.572794+0000","last_unstale":"2024-09-11T05:28:30.064774+0000","last_undegraded":"2024-09-11T05:28:30.064774+0000","last_fullsized":"2024-09-11T05:28:30.064774+0000","mapping_epoch":66,"log_start":"45'9600","ondisk_log_start":"45'9600","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19654","last_scrub_stamp":"2024-09-11T05:28:30.064501+0000","last_deep_scrub":"49'19654","last_deep_scrub_stamp":"2024-09-11T05:28:30.064501+0000","last_clean_scrub_stamp":"2024-09-11T05:28:30.064501+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:27.809350+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3498,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:26.241522+0000","last_change":"2024-09-11T05:28:26.241522+0000","last_active":"2024-09-11T05:28:26.241522+0000","last_peered":"2024-09-11T05:28:26.241522+0000","last_clean":"2024-09-11T05:28:26.241522+0000","last_became_active":"2024-09-11T05:13:22.497585+0000","last_became_peered":"2024-09-11T05:13:22.497585+0000","last_unstale":"2024-09-11T05:28:26.241522+0000","last_undegraded":"2024-09-11T05:28:26.241522+0000","last_fullsized":"2024-09-11T05:28:26.241522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:28:26.241331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:28:26.241331+0000","last_clean_scrub_stamp":"2024-09-11T05:28:26.241331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:27.973177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19800","reported_seq":19984,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:32.098342+0000","last_change":"2024-09-11T05:28:32.098342+0000","last_active":"2024-09-11T05:28:32.098342+0000","last_peered":"2024-09-11T05:28:32.098342+0000","last_clean":"2024-09-11T05:28:32.098342+0000","last_became_active":"2024-09-11T05:26:11.920133+0000","last_became_peered":"2024-09-11T05:26:11.920133+0000","last_unstale":"2024-09-11T05:28:32.098342+0000","last_undegraded":"2024-09-11T05:28:32.098342+0000","last_fullsized":"2024-09-11T05:28:32.098342+0000","mapping_epoch":58,"log_start":"45'9700","ondisk_log_start":"45'9700","created":15,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19800","last_scrub_stamp":"2024-09-11T05:28:32.098174+0000","last_deep_scrub":"49'19800","last_deep_scrub_stamp":"2024-09-11T05:28:32.098174+0000","last_clean_scrub_stamp":"2024-09-11T05:28:32.098174+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:42:02.291778+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16558,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:24.079056+0000","last_change":"2024-09-11T05:28:24.079056+0000","last_active":"2024-09-11T05:28:24.079056+0000","last_peered":"2024-09-11T05:28:24.079056+0000","last_clean":"2024-09-11T05:28:24.079056+0000","last_became_active":"2024-09-11T05:14:13.982832+0000","last_became_peered":"2024-09-11T05:14:13.982832+0000","last_unstale":"2024-09-11T05:28:24.079056+0000","last_undegraded":"2024-09-11T05:28:24.079056+0000","last_fullsized":"2024-09-11T05:28:24.079056+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:28:24.078969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:28:24.078969+0000","last_clean_scrub_stamp":"2024-09-11T05:28:24.078969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:33.409766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018356799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:31.070616+0000","last_change":"2024-09-11T05:28:31.070616+0000","last_active":"2024-09-11T05:28:31.070616+0000","last_peered":"2024-09-11T05:28:31.070616+0000","last_clean":"2024-09-11T05:28:31.070616+0000","last_became_active":"2024-09-11T05:14:13.749572+0000","last_became_peered":"2024-09-11T05:14:13.749572+0000","last_unstale":"2024-09-11T05:28:31.070616+0000","last_undegraded":"2024-09-11T05:28:31.070616+0000","last_fullsized":"2024-09-11T05:28:31.070616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T05:28:31.070457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T05:28:31.070457+0000","last_clean_scrub_stamp":"2024-09-11T05:28:31.070457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:36.145101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00239858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"67'19591","reported_seq":19789,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T05:28:28.107470+0000","last_change":"2024-09-11T05:28:28.107470+0000","last_active":"2024-09-11T05:28:28.107470+0000","last_peered":"2024-09-11T05:28:28.107470+0000","last_clean":"2024-09-11T05:28:28.107470+0000","last_became_active":"2024-09-11T05:25:09.923296+0000","last_became_peered":"2024-09-11T05:25:09.923296+0000","last_unstale":"2024-09-11T05:28:28.107470+0000","last_undegraded":"2024-09-11T05:28:28.107470+0000","last_fullsized":"2024-09-11T05:28:28.107470+0000","mapping_epoch":54,"log_start":"45'9500","ondisk_log_start":"45'9500","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'19591","last_scrub_stamp":"2024-09-11T05:28:28.107303+0000","last_deep_scrub":"67'19591","last_deep_scrub_stamp":"2024-09-11T05:28:28.107303+0000","last_clean_scrub_stamp":"2024-09-11T05:28:28.107303+0000","objects_scrubbed":1,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.982429+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16597,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3519,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281694,"num_write_kb":489677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42512,"num_bytes_recovered":30318592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43683840,"data_stored":43676882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986365,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334896,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":94036944,"statfs":{"total":96636764160,"available":96293830656,"internally_reserved":0,"allocated":557056,"data_stored":188036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342355627},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659310,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505656,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866184,"statfs":{"total":96636764160,"available":96118972416,"internally_reserved":0,"allocated":565248,"data_stored":188672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":517205675},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":24,"seq":103079215298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502420,"kb_used_data":43204,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93869420,"statfs":{"total":96636764160,"available":96122286080,"internally_reserved":0,"allocated":44240896,"data_stored":43864320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":43679744,"data_stored":43676265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T05:28:44.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-11T05:28:44.733 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-11T05:28:44.733 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-11T05:28:44.734 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T05:28:44.734 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:28:47.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-11T05:28:50.835 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T05:28:50.835 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T05:28:50.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:28:50.935 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T05:28:50.935 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T05:28:50.936 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:28:52.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-11T05:28:52.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-11T05:28:57.037 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T05:28:57.037 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-11T05:28:57.038 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-11T05:28:57.038 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:28:57.068 INFO:tasks.ceph.mgr.x:Stopped 2024-09-11T05:28:57.068 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-11T05:28:57.068 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-11T05:28:57.069 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:28:57.134 INFO:tasks.ceph.mon.a:Stopped 2024-09-11T05:28:57.134 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-11T05:28:57.134 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:28:57.191 INFO:tasks.ceph.mon.b:Stopped 2024-09-11T05:28:57.191 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-11T05:28:57.192 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T05:28:57.249 INFO:tasks.ceph.mon.c:Stopped 2024-09-11T05:28:57.249 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-11T05:28:57.250 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-11T05:28:57.424 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:28:57.424 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-11T05:28:57.838 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-11T05:28:58.150 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:28:58.150 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-11T05:28:58.360 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:28:58.360 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-11T05:28:58.504 INFO:tasks.ceph:Archiving mon data... 2024-09-11T05:28:58.504 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194/data/mon.a.tgz 2024-09-11T05:28:58.505 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-11T05:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.DDRQS5tuIz 2024-09-11T05:28:58.525 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.DDRQS5tuIz 2024-09-11T05:28:58.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.DDRQS5tuIz 2024-09-11T05:28:58.766 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.DDRQS5tuIz is 306KB 2024-09-11T05:28:58.825 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.DDRQS5tuIz 2024-09-11T05:28:58.842 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194/data/mon.b.tgz 2024-09-11T05:28:58.843 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-11T05:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.KmKbFt9AK8 2024-09-11T05:28:58.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KmKbFt9AK8 2024-09-11T05:28:59.008 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.KmKbFt9AK8 2024-09-11T05:28:59.071 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.KmKbFt9AK8 is 312KB 2024-09-11T05:28:59.129 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.KmKbFt9AK8 2024-09-11T05:28:59.134 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194/data/mon.c.tgz 2024-09-11T05:28:59.134 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-11T05:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.FnFAeJMnO4 2024-09-11T05:28:59.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.FnFAeJMnO4 2024-09-11T05:28:59.304 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.FnFAeJMnO4 2024-09-11T05:28:59.367 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.FnFAeJMnO4 is 312KB 2024-09-11T05:28:59.429 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.FnFAeJMnO4 2024-09-11T05:28:59.434 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-11T05:28:59.434 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T05:28:59.552 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T05:28:59.553 INFO:tasks.ceph:Archiving crash dumps... 2024-09-11T05:28:59.554 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194/remote/smithi133/crash 2024-09-11T05:28:59.555 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T05:28:59.608 INFO:tasks.ceph:Compressing logs... 2024-09-11T05:28:59.608 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T05:28:59.672 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2024-09-11T05:28:59.673 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41753.log 2024-09-11T05:28:59.673 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35457.log 2024-09-11T05:28:59.673 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz 2024-09-11T05:28:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41753.log.gz 2024-09-11T05:28:59.674 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30620.log 2024-09-11T05:28:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34256.log 2024-09-11T05:28:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35457.log: /var/log/ceph/ceph-client.admin.30620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39645.log 2024-09-11T05:28:59.675 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35457.log.gz 2024-09-11T05:28:59.675 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30620.log.gz 2024-09-11T05:28:59.675 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39359.log 2024-09-11T05:28:59.675 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34256.log.gz 2024-09-11T05:28:59.676 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39332.log 2024-09-11T05:28:59.677 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39645.log.gz 2024-09-11T05:28:59.677 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39359.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39359.log.gz -5 2024-09-11T05:28:59.677 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30194.log 2024-09-11T05:28:59.677 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30084.log 2024-09-11T05:28:59.677 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39332.log.gz 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40711.log 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30194.log.gz 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46568.log 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30084.log.gz 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41493.log 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40711.log.gz 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41285.log 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46568.log.gz 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2024-09-11T05:28:59.679 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41493.log.gz 2024-09-11T05:28:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 2024-09-11T05:28:59.682 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41285.log.gz 2024-09-11T05:28:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41909.log 2024-09-11T05:28:59.682 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2024-09-11T05:28:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-09-11T05:28:59.682 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2024-09-11T05:28:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51200.log 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41909.log.gz 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43212.log 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gz 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51773.log 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51200.log.gz 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47924.log 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43212.log.gz 2024-09-11T05:28:59.683 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51773.log.gz 2024-09-11T05:28:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39801.log 2024-09-11T05:28:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43524.log 2024-09-11T05:28:59.685 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47924.log.gz 2024-09-11T05:28:59.685 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46021.log 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39801.log.gz 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31148.log 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43524.log.gz 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30282.log 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46021.log.gz 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31148.log.gz 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30282.log.gz 2024-09-11T05:28:59.686 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-09-11T05:28:59.690 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45318.log 2024-09-11T05:28:59.690 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-09-11T05:28:59.690 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34673.log 2024-09-11T05:28:59.690 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-09-11T05:28:59.690 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29150.log 2024-09-11T05:28:59.690 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45318.log.gz 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42143.log 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34673.log.gz 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29150.log.gz 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35237.log 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42143.log.gz 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49510.log 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39124.log 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35237.log.gz 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46880.log 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49510.log.gz 2024-09-11T05:28:59.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38262.log 2024-09-11T05:28:59.692 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39124.log.gz 2024-09-11T05:28:59.692 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44642.log 2024-09-11T05:28:59.692 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46880.log.gz 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38262.log.gz 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37300.log 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44642.log.gz 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29799.log 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35193.log 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37300.log.gz 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34646.log 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29799.log.gz 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49458.log 2024-09-11T05:28:59.703 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35193.log.gz 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34646.log.gz 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51669.log 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49458.log.gz 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46099.log 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34334.log 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51669.log.gz 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42221.log 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46099.log.gz 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34334.log.gz 2024-09-11T05:28:59.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42221.log.gz 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45162.log 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29952.log 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45162.log.gz 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30598.log 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29952.log.gz 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43784.log 2024-09-11T05:28:59.705 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30598.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37586.log 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49094.log 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43784.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38522.log 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37586.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49094.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38782.log 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38522.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50264.log 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38782.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2024-09-11T05:28:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50264.log.gz 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48470.log 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30730.log 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45058.log 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48470.log.gz 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39879.log 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30730.log.gz 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45058.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38574.log 2024-09-11T05:28:59.707 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45058.log.gz 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50940.log 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39879.log.gz 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44954.log 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38574.log.gz 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50940.log.gz 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47767.log 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44954.log: /var/log/ceph/ceph-client.admin.48522.log 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44954.log.gz 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49900.log 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47767.log.gz 2024-09-11T05:28:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27994.log 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48522.log.gz 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30972.log 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49900.log.gz 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.27994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30018.log 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27994.log.gz 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30549.log 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30972.log.gz 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37170.log 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30018.log.gz 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30549.log.gz 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48860.log 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41831.log 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37170.log.gz 2024-09-11T05:28:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46438.log 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48860.log.gz 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41831.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.41831.log.gz 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.41701.log 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37820.log 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46438.log.gz 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45292.log 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41701.log.gz 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34438.log 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37820.log.gz 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45292.log: -- /var/log/ceph/ceph-client.admin.35017.log 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45292.log.gz 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34516.log 2024-09-11T05:28:59.723 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34438.log.gz 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48990.log 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35017.log.gz 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42432.log 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34516.log.gz 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28845.log 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48990.log.gz 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34676.log 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42432.log.gz 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45370.log 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28845.log.gz 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43134.log 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34676.log.gz 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45370.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.29232.log 2024-09-11T05:28:59.724 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45370.log.gz 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29235.log 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43134.log.gz 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47088.log 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29232.log.gz 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38887.log 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29235.log.gz 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31060.log 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47088.log.gz 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34074.log 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38887.log.gz 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30796.log 0.0% 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31060.log.gz 2024-09-11T05:28:59.725 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48834.log 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34074.log.gz 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38054.log 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30796.log.gz 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48834.log.gz 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40217.log 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38054.log.gz 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51408.log 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40451.log 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40217.log.gz 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30260.log 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51408.log.gz 2024-09-11T05:28:59.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32083.log 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40451.log.gz 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42406.log 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30260.log.gz 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30752.log 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32083.log.gz 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45006.log 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42406.log.gz 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49744.log 2024-09-11T05:28:59.727 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30752.log.gz 2024-09-11T05:28:59.728 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35479.log 2024-09-11T05:28:59.728 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45006.log.gz 2024-09-11T05:28:59.728 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38913.log 2024-09-11T05:28:59.728 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49744.log.gz 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47429.log 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35479.log.gz 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28924.log 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38913.log.gz 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46360.log 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47429.log.gz 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47377.log 2024-09-11T05:28:59.729 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28924.log.gz 2024-09-11T05:28:59.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33866.log 2024-09-11T05:28:59.768 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46360.log.gz 2024-09-11T05:28:59.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45474.log 2024-09-11T05:28:59.768 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47377.log.gz 2024-09-11T05:28:59.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32082.log 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33866.log.gz 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.45474.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41207.log 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45474.log.gz 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35127.log 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32082.log.gz 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50290.log 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41207.log.gz 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45864.log 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35127.log.gz 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38444.log 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50290.log.gz 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48314.log 2024-09-11T05:28:59.769 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45864.log.gz 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43940.log 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38444.log.gz 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43290.log 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48314.log.gz 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35105.log 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43940.log.gz 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41883.log 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43290.log.gz 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45396.log 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35105.log.gz 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46073.log 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41883.log.gz 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30686.log 2024-09-11T05:28:59.770 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45396.log.gz 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46073.log.gz 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50628.log 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30686.log.gz 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47872.log 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52007.log 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50628.log.gz 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47872.log.gz 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35347.log 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30436.log 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52007.log.gz 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49718.log 2024-09-11T05:28:59.771 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35347.log.gz 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42039.log 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30436.log.gz 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45994.log 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49718.log.gz 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42299.log 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42039.log.gz 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44798.log 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51799.log 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42299.log.gz 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44798.log.gz 2024-09-11T05:28:59.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34178.log 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51799.log.gz 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42770.log 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34178.log.gz 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32027.log 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42770.log.gz 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43732.log 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32027.log.gz 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36412.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-09-11T05:28:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42169.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43732.log.gz 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40477.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46125.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35501.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40477.log.gz 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28812.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46125.log.gz 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35501.log.gz 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28812.log.gz 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33634.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51434.log 2024-09-11T05:28:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45682.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33634.log.gz 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48340.log 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51434.log.gz 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.49406.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44590.log 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40841.log 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48340.log.gz 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42091.log 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44590.log.gz 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.40841.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38184.log 2024-09-11T05:28:59.775 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40841.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42091.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36698.log 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38184.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36985.log 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36698.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47325.log 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28714.log 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36985.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30392.log 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47325.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28714.log.gz 2024-09-11T05:28:59.776 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44824.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47062.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30392.log.gz 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44824.log.gz 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34230.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42195.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47062.log.gz 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34230.log.gz 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45500.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39749.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42195.log.gz 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45500.log.gz 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41727.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48652.log 2024-09-11T05:28:59.777 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39749.log.gz 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41727.log.gz 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51877.log 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44902.log 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48652.log.gz 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51877.log.gz 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39463.log 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43628.log 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44902.log.gz 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39463.log.gz 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47637.log 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43628.log.gz 2024-09-11T05:28:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47637.log.gz 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43914.log 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43914.log.gz 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44200.log 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46594.log 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44200.log.gz 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28994.log 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46594.log.gz 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz 2024-09-11T05:28:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46776.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28994.log.gz 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37768.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45578.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46776.log.gz 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37768.log.gz 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45266.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45578.log.gz 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30708.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49380.log 2024-09-11T05:28:59.780 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45266.log.gz 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30708.log.gz 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30862.log 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45916.log 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49380.log.gz 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43108.log 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30862.log.gz 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45916.log.gz 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36959.log 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48418.log 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43108.log.gz 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39046.log 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36959.log.gz 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.48418.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33403.log 2024-09-11T05:28:59.781 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48418.log.gz 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34204.log 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39046.log.gz 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46542.log 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33403.log.gz 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42013.log 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34204.log.gz 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29514.log 2024-09-11T05:28:59.782 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46542.log.gz 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51538.log 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42013.log.gz 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36776.log 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29514.log.gz 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37690.log 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51538.log.gz 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44460.log 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36776.log.gz 2024-09-11T05:28:59.784 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40087.log 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37690.log.gz 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50914.log 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44460.log.gz 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40373.log 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40087.log.gz 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41649.log 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50914.log.gz 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48132.log 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40373.log.gz 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37274.log 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41649.log.gz 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39541.log 2024-09-11T05:28:59.785 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48132.log.gz 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46828.log 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37274.log.gz 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37950.log 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39541.log.gz 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46828.log.gz 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43862.log 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37950.log.gz 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42562.log 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34973.log 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43862.log.gz 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39853.log 2024-09-11T05:28:59.786 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42562.log.gz 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38080.log 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34973.log.gz 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39853.log.gz 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30238.log 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34152.log 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38080.log.gz 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30238.log.gz 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42510.log 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47976.log 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34152.log.gz 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38730.log 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42510.log.gz 2024-09-11T05:28:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44746.log 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47976.log.gz 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29877.log 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38730.log.gz 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50056.log 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44746.log.gz 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29877.log.gz 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50056.log.gz 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51903.log 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49822.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.35526.log -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-09-11T05:28:59.788 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48964.log 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51903.log.gz 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48808.log 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35526.log.gz 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29647.log 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48964.log.gz 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37560.log 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48808.log.gz 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29647.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.29647.log.gz 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32313.log 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33712.log 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37560.log.gz 2024-09-11T05:28:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32313.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32313.log.gz -5 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44538.log 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33712.log.gz 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43498.log 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50966.log 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44538.log.gz 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43498.log.gz 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39827.log 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50966.log.gz 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50706.log 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38418.log 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39827.log.gz 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50706.log.gz 2024-09-11T05:28:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-09-11T05:28:59.791 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50680.log 2024-09-11T05:28:59.791 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38418.log.gz 2024-09-11T05:28:59.791 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2024-09-11T05:28:59.791 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-09-11T05:28:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30458.log 2024-09-11T05:28:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50680.log.gz 2024-09-11T05:28:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37976.log 2024-09-11T05:28:59.941 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-09-11T05:28:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30458.log.gz 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30040.log 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38756.log 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37976.log.gz 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30040.log.gz 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45084.log 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46516.log 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38756.log.gz 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45084.log.gz 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42978.log 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46516.log.gz 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37872.log 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-09-11T05:28:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37404.log 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42978.log.gz 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37872.log.gz 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37404.log.gz 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46854.log 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33764.log 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33351.log 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46854.log.gz 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30414.log 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33764.log.gz 2024-09-11T05:28:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33351.log.gz 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47010.log 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30414.log.gz 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42380.log 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47010.log.gz 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49146.log 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39306.log 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42380.log.gz 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49146.log.gz 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39098.log 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39437.log 2024-09-11T05:28:59.944 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39306.log.gz 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39098.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.48912.log 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39098.log.gz 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40165.log 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39437.log.gz 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37846.log 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40165.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.40165.log.gz /var/log/ceph/ceph-client.admin.25116.log 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47481.log 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37846.log.gz 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.25116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40607.log 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25116.log.gz 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-09-11T05:28:59.945 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47481.log.gz 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30348.log 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40607.log.gz 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47715.log 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47273.log 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30348.log.gz 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29124.log 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47715.log.gz 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28760.log 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47273.log.gz 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29124.log.gz 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38496.log 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-09-11T05:28:59.946 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28760.log.gz 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38496.log.gz 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34568.log 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39775.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.43836.log 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29543.log 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34568.log.gz 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43836.log.gz 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47689.log 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29543.log.gz 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31170.log 2024-09-11T05:28:59.947 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40555.log 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47689.log.gz 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31170.log.gz 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50472.log 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41467.log 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40555.log.gz 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30370.log 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50472.log.gz 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47611.log 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41467.log.gz 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39567.log 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30370.log.gz 2024-09-11T05:28:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50160.log 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47611.log.gz 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29996.log 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39567.log.gz 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28923.log 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50160.log.gz 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45604.log 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29996.log.gz 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50836.log 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28923.log.gz 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45604.log.gz 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-09-11T05:28:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42900.log 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36750.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz -- 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.47663.log 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29488.log 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42900.log.gz 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51616.log 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47663.log.gz 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29392.log 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29488.log.gz 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39150.log 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51616.log.gz 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50108.log 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29392.log.gz 2024-09-11T05:28:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39593.log 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39150.log.gz 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41779.log 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50108.log.gz 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42273.log 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39593.log.gz 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49042.log 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41779.log.gz 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42273.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42273.log.gz 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44694.log 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51044.log 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49042.log.gz 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39957.log 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44694.log.gz 2024-09-11T05:28:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51044.log.gz 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35215.log 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39957.log.gz 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30128.log 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gz 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41129.log 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35215.log.gz 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35149.log 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30128.log.gz 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41935.log 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41129.log.gz 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40919.log 2024-09-11T05:28:59.952 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35149.log.gz 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46646.log 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41935.log.gz 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45240.log 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40919.log.gz 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46646.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46646.log.gz 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36672.log 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43550.log 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45240.log.gz 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30062.log 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36672.log.gz 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43550.log.gz 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36620.log 2024-09-11T05:28:59.953 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30062.log.gz 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50446.log 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36620.log.gz 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51174.log 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36854.log 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50446.log.gz 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51174.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.51174.log.gz 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36594.log 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29313.log 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36854.log.gz 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36594.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36594.log.gz 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/ceph-client.admin.25262.log 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38132.log 2024-09-11T05:28:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29313.log.gz 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.25262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49874.log 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25262.log.gz 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48184.log 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38132.log.gz 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50498.log 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49874.log.gz 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48184.log.gz 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34100.log 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50498.log.gz 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33322.log 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34100.log.gz 2024-09-11T05:28:59.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45526.log 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39983.log 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33322.log.gz 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34906.log 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45526.log.gz 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51695.log 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39983.log.gz 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29236.log 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34906.log.gz 2024-09-11T05:28:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50368.log 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51695.log.gz 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32261.log 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29236.log.gz 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50368.log.gz 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32261.log.gz 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35391.log 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49328.log 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-09-11T05:28:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37898.log 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30774.log 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49328.log.gz 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31016.log 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37898.log.gz 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29384.log 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30774.log.gz 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34542.log 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31016.log.gz 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38861.log 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29384.log.gz 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38366.log 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34542.log.gz 2024-09-11T05:28:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38861.log.gz 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30928.log 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38366.log.gz 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51252.log 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30928.log.gz 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49484.log 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51252.log.gz 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42614.log 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48002.log 0.0% 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49484.log.gz 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-09-11T05:28:59.959 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42614.log.gz 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33892.log 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48002.log.gz 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49952.log 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33892.log: -- /var/log/ceph/ceph-client.admin.39385.log 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33892.log.gz 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38548.log 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49952.log.gz 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39385.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.39385.log.gz -- 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.35281.log 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29003.log 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38548.log.gz 2024-09-11T05:28:59.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38314.log 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35281.log.gz 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29003.log.gz 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41051.log 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43706.log 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38314.log.gz 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34880.log 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41051.log.gz 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43706.log.gz 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34880.log.gz 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47846.log 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-09-11T05:28:59.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46724.log 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40425.log 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47846.log.gz 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40321.log 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46724.log.gz 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40425.log.gz 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48392.log 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40321.log.gz 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50316.log 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47351.log 2024-09-11T05:28:59.962 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48392.log.gz 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50316.log.gz 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47351.log.gz 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44226.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz -- 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.44616.log 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39254.log 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37742.log 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39254.log.gz 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29202.log 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37612.log 2024-09-11T05:28:59.963 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37742.log.gz 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29202.log.gz 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40737.log 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37612.log.gz 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30575.log 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43576.log 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40737.log.gz 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38210.log 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30575.log.gz 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40113.log 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43576.log.gz 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29618.log 2024-09-11T05:28:59.964 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38210.log.gz 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51486.log 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40113.log.gz 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25467.log 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29618.log.gz 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34490.log 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51486.log.gz 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.25467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50602.log 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25467.log.gz 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40815.log 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34490.log.gz 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38965.log 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50602.log.gz 2024-09-11T05:28:59.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48444.log 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40815.log.gz 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38965.log.gz 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35325.log 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46672.log 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48444.log.gz 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49224.log 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35325.log.gz 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45734.log 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46672.log.gz 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37430.log 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49224.log.gz 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41155.log 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45734.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45734.log.gz 2024-09-11T05:28:59.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37352.log 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37430.log.gz 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44018.log 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41155.log.gz 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51721.log 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37352.log.gz 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29851.log 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44928.log 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51721.log.gz 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45110.log 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29851.log.gz 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33296.log 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44928.log.gz 2024-09-11T05:28:59.967 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45110.log: /var/log/ceph/ceph-client.admin.29314.log 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45110.log.gz 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51278.log 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33296.log.gz 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29314.log.gz 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39020.log 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30216.log 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51278.log.gz 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42848.log 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39020.log.gz 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30216.log.gz 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25341.log 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42848.log.gz 2024-09-11T05:28:59.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.25341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39515.log 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25341.log.gz 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44382.log 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33608.log 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39515.log.gz 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44382.log: 8.7% -- replaced with /var/log/ceph/ceph-client.admin.35391.log.gz 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41805.log 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44382.log.gz 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33608.log.gz 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.41805.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44174.log 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41805.log.gz 2024-09-11T05:28:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28734.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44174.log: /var/log/ceph/ceph-client.admin.40191.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44174.log.gz 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28734.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28734.log.gz 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31038.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40191.log.gz 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51643.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45968.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31038.log.gz 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51955.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51643.log.gz 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41233.log 2024-09-11T05:28:59.970 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45968.log.gz 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34620.log 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51955.log.gz 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41233.log.gz 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49250.log 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34620.log.gz 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49250.log.gz 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29751.log 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32053.log 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35369.log 2024-09-11T05:28:59.971 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29751.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32053.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49016.log 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29176.log 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35369.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49016.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51825.log 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29176.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43056.log 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48574.log 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51825.log: /var/log/ceph/ceph-client.admin.43056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51825.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43056.log.gz 2024-09-11T05:28:59.972 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48678.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48574.log.gz 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43420.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.48678.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48678.log.gzgzip 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48938.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25365.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43420.log.gz 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33530.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48938.log.gz 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.25365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25365.log.gz 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34282.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44564.log 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33530.log.gz 2024-09-11T05:28:59.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30106.log 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34282.log.gz 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38288.log 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44564.log.gz 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42117.log 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30106.log.gz 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38236.log 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38288.log.gz 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41441.log 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42117.log.gz 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29699.log 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38236.log.gz 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51981.log 2024-09-11T05:28:59.974 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41441.log.gz 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30304.log 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29699.log.gz 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41337.log 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51981.log.gz 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30304.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30304.log.gz 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32287.log 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41337.log.gz 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47169.log 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32287.log.gz 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37089.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50238.log 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46177.log 2024-09-11T05:28:59.975 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47169.log.gz 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38835.log 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50238.log.gz 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46177.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.49796.log -- replaced with /var/log/ceph/ceph-client.admin.46177.log.gz 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37456.log 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38835.log.gz 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49796.log.gz 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29072.log 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37456.log.gz 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46958.log 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz 2024-09-11T05:28:59.976 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29072.log.gz 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr: 95.7% -- replaced with /var/log/ceph/ceph.log.gz/var/log/ceph/ceph-client.admin.46958.log: 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31082.log 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46958.log.gz 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48210.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz -5 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40633.log 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31082.log.gz 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48054.log 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38470.log 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40633.log.gz 2024-09-11T05:28:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48054.log.gz 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40867.log 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33556.log 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38470.log.gz 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47299.log 0.0% 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40867.log.gz 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40997.log 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33556.log.gz 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37924.log 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47299.log.gz 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30524.log 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40997.log.gz 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40685.log 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2024-09-11T05:28:59.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38392.log 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30524.log.gz 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40685.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40685.log.gz -5 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50420.log 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43368.log 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38392.log.gz 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46984.log 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50420.log.gz 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43368.log.gzgzip 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45760.log 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47794.log 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46984.log.gz 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40503.log 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45760.log.gz 2024-09-11T05:28:59.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47794.log.gz 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28846.log 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40503.log.gz 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.28846.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48496.log 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28846.log.gz 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32235.log 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28916.log 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48496.log.gz 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49198.log 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32235.log.gz 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28916.log.gz 2024-09-11T05:28:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30818.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38808.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49198.log.gz 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34360.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30818.log.gz 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38808.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38808.log.gz 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47403.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34360.log.gz 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33816.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37794.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47403.log.gz 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33816.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.33816.log.gz /var/log/ceph/ceph-client.admin.48028.log 2024-09-11T05:28:59.981 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37794.log.gz 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43160.log 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48028.log.gz 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42065.log 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50888.log 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43160.log.gz 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5/var/log/ceph/ceph-client.admin.42065.log: --verbose -- /var/log/ceph/ceph-client.admin.49276.log 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42065.log.gz 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29803.log 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50888.log.gz 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47741.log 2024-09-11T05:28:59.982 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49276.log.gz 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46620.log 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29803.log.gz 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47559.log 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47741.log.gz 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51018.log 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46620.log.gz 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43186.log 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47559.log.gz 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49172.log 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51018.log.gz 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33452.log 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43186.log.gz 2024-09-11T05:28:59.983 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32135.log 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49172.log.gz 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51929.log 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33452.log.gz 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32135.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.32135.log.gz -5 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39176.log 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49614.log 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51929.log.gz 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33664.log 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39176.log.gz 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40009.log 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49614.log.gz 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33664.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.33664.log.gz --verbose 2024-09-11T05:28:59.984 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/ceph-client.admin.44850.log 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41857.log 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40009.log.gz 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41987.log 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44850.log.gz 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41857.log.gz 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29098.log 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41987.log.gz 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gz 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40399.log 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29098.log.gz 2024-09-11T05:28:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47820.log 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46308.log 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40399.log.gz 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47820.log.gz 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50030.log 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46308.log.gz 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50030.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21502.log 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50030.log.gz 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35413.log 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.tmp-client.admin.21502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30884.log 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21502.log.gz 2024-09-11T05:28:59.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34386.log 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35413.log.gz 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40035.log 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30884.log.gz 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40269.log 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34386.log.gz 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41311.log 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40035.log.gz 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40269.log.gz 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51382.log 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45032.log 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41311.log.gz 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45812.log 2024-09-11T05:28:59.987 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51382.log.gz 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34464.log 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45032.log.gz 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40243.log 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45812.log.gz 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40971.log 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34464.log.gz 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29306.log 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40243.log.gz 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41181.log 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40971.log.gz 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29306.log.gz 2024-09-11T05:28:59.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43316.log 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41181.log.gz 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47950.log 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43316.log.gz 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.45552.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43342.log 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48236.log 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47950.log.gz 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46932.log 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43342.log.gz 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50992.log 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48236.log.gz 2024-09-11T05:28:59.989 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40061.log 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46932.log.gz 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39202.log 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50992.log.gz 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34022.log 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40061.log.gz 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37141.log 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39202.log.gz 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28664.log 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34022.log.gz 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5/var/log/ceph/ceph-client.admin.37141.log: --verbose -- /var/log/ceph/ceph-client.admin.40945.log 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37141.log.gz 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33944.log 2024-09-11T05:28:59.990 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28664.log.gz 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41597.log 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40945.log.gz 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40893.log 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33944.log.gz 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43446.log 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41597.log.gz 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40893.log.gz 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29569.log 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50862.log 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43446.log.gz 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29569.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29569.log.gz 2024-09-11T05:28:59.991 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28844.log 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50862.log.gz 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47143.log 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45188.log 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28844.log.gz 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36906.log 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47143.log.gz 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45890.log 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45188.log.gz 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29974.log 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36906.log.gz 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36802.log 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45890.log.gz 2024-09-11T05:28:59.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41103.log 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29974.log.gz 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5/var/log/ceph/ceph-client.admin.36802.log: --verbose -- /var/log/ceph/ceph-client.admin.35259.log 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36802.log.gz 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42796.log 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41103.log.gz 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43654.log 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35259.log.gz 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48288.log 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42796.log.gz 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42247.log 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43654.log.gz 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48288.log.gz 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2024-09-11T05:28:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42247.log.gz 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31104.log 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46490.log 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50082.log 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31104.log.gz 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25400.log 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46490.log.gz 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42744.log 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50082.log.gz 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.25400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25400.log.gz 2024-09-11T05:28:59.994 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38002.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42744.log.gz 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43472.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38002.log.gz 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28689.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41545.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43472.log.gz 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50004.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28689.log.gz 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50212.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41545.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41545.log.gz 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40789.log 2024-09-11T05:28:59.995 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50004.log.gz 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49926.log 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50212.log.gz 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40789.log.gz 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33996.log 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28998.log 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49926.log.gz 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39671.log 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33996.log.gz 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40763.log 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28998.log.gz 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33970.log 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39671.log.gz 2024-09-11T05:28:59.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34126.log 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40763.log.gz 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48158.log 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33970.log.gz 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34126.log.gz 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43030.log 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48158.log.gz 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39931.log 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43238.log 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43030.log.gz 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32131.log 2024-09-11T05:28:59.997 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39931.log.gz 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47533.log 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43238.log.gz 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45942.log 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32131.log.gz 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34412.log 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47533.log.gz 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45214.log 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45942.log.gz 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44044.log 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34412.log.gz 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35303.log 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45214.log.gz 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50576.log 2024-09-11T05:28:59.998 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44044.log.gz 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42588.log 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35303.log.gz 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34776.log 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50576.log.gz 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50394.log 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42588.log.gz 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46698.log 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34776.log.gz 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33768.log 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50394.log.gz 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46698.log.gz 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-09-11T05:28:59.999 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33768.log.gz 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39411.log 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48730.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33404.log 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30326.log 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39411.log.gz 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33404.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.33404.log.gz 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.30994.log 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30326.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30326.log.gz 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46047.log 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30994.log.gz 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37196.log 2024-09-11T05:29:00.000 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46047.log.gz 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46750.log 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43264.log 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37196.log.gz 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49588.log 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46750.log.gz 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46282.log 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43264.log.gz 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44148.log 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49588.log.gz 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46256.log 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46282.log.gz 2024-09-11T05:29:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40347.log 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44148.log.gz 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32183.log 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46256.log.gz 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36880.log 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40347.log.gz 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48704.log 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32183.log.gz 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30950.log 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36880.log.gz 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34995.log 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48704.log.gz 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29929.log 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30950.log.gz 2024-09-11T05:29:00.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37482.log 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34995.log.gz 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37115.log 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29929.log.gz 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48106.log 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37482.log.gz 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38106.log 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37115.log.gz 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51564.log 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48106.log.gz 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45344.log 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38106.log.gz 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43758.log 2024-09-11T05:29:00.003 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51564.log.gz 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45344.log.gz 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40581.log 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41077.log 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43758.log.gz 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41675.log 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40581.log.gz 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34828.log 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41077.log.gz 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45136.log 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43004.log 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34828.log.gz 2024-09-11T05:29:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48262.log 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45136.log.gz 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51851.log 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43004.log.gz 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38340.log 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48262.log.gz 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32339.log 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51851.log.gz 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38626.log 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38340.log.gz 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34929.log 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32339.log.gz 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33716.log 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38626.log.gz 2024-09-11T05:29:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44330.log 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34929.log.gz 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40659.log 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33716.log.gz 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44330.log.gz 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39280.log 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42325.log 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40659.log.gz 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39280.log.gz 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35061.log 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42325.log.gz 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5/var/log/ceph/ceph-client.admin.43810.log: --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2024-09-11T05:29:00.006 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35061.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35061.log.gz 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.43966.log 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41571.log 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39723.log 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43966.log.gz 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34594.log 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41571.log.gz 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29462.log 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39723.log.gz 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44252.log 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34594.log.gz 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46334.log 2024-09-11T05:29:00.007 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29462.log.gz 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44252.log.gz 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38994.log 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46334.log.gz 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42458.log 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38994.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38994.log.gz 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37664.log 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34308.log 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42458.log.gz 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35083.log 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37664.log.gz 2024-09-11T05:29:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47036.log 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34308.log.gz 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41623.log 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35083.log.gz 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33352.log 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47036.log.gz 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47507.log 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41623.log.gz 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33352.log.gz 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47507.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47507.log.gz 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47455.log 2024-09-11T05:29:00.009 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37534.log 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.47455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50654.log 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47455.log.gz 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30664.log 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37534.log: /var/log/ceph/ceph-client.admin.50654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37534.log.gz 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50654.log.gz 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45630.log 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37011.log 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30664.log.gz 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34048.log 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45630.log.gz 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44876.log 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37011.log.gz 2024-09-11T05:29:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30480.log 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34048.log.gz 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30840.log 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44876.log.gz 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28786.log 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30480.log.gz 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29388.log 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30840.log.gz 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.28786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38704.log 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28786.log.gz 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46203.log 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29388.log.gz 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49536.log 2024-09-11T05:29:00.011 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38704.log.gz 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46203.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.46203.log.gz /var/log/ceph/ceph-client.admin.42351.log 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49432.log 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49536.log.gz 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50342.log 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42351.log.gz 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33663.log 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49432.log.gz 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48782.log 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50342.log.gz 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43394.log 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33663.log.gz 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44278.log 2024-09-11T05:29:00.012 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48782.log.gz 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30642.log 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43394.log.gz 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44278.log.gz 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44668.log 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33456.log 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30642.log.gz 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44096.log 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44668.log.gz 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37378.log 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33456.log.gz 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44096.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44096.log.gz 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32187.log 2024-09-11T05:29:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41389.log 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37378.log.gz 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.32187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35171.log 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32187.log.gz 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41389.log.gz 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35171.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35171.log.gz 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38678.log 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42822.log: 95.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36933.log 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-09-11T05:29:00.014 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50810.log 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38678.log.gz 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30150.log 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36933.log.gz 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39228.log 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50810.log.gz 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34800.log 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30150.log.gz 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51356.log 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39228.log.gz 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51070.log 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34800.log.gz 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46464.log 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51356.log.gz 2024-09-11T05:29:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25317.log 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51070.log.gz 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46464.log.gz 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49562.log 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.25317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41961.log 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25317.log.gz 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49120.log 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49562.log.gz 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29646.log 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41961.log.gz 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30172.log 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49120.log.gz 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29646.log.gz 2024-09-11T05:29:00.016 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38600.log 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30172.log.gz 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50134.log 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44720.log 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38600.log.gz 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46230.log 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50134.log.gz 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31126.log 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44720.log.gz 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37508.log 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46230.log.gz 2024-09-11T05:29:00.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.31126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47898.log 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31126.log.gz 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33582.log 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37508.log.gz 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33840.log 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47898.log.gz 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29747.log 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33582.log.gz 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33840.log.gz 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42926.log 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29747.log.gz 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37037.log 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2024-09-11T05:29:00.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42484.log 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42926.log.gz 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51096.log 0.0% 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37037.log.gz 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44486.log 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42484.log.gz 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51096.log.gz 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44772.log 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44486.log.gz 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36542.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz -5 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30906.log 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34951.log 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44772.log.gz 2024-09-11T05:29:00.019 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44304.log 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30906.log.gz 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49068.log 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34951.log.gz 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41259.log 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44304.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44304.log.gz 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49068.log.gz 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41259.log.gz 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36828.log 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2024-09-11T05:29:00.020 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36828.log.gz 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34854.log 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45422.log 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42718.log 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34854.log.gz 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29903.log 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45422.log.gz 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30502.log 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42718.log.gz 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39619.log 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29903.log.gz 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.30502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40139.log 2024-09-11T05:29:00.021 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30502.log.gz 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39905.log 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39619.log.gz 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39697.log 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40139.log.gz 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37248.log 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39905.log.gz 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47585.log 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39697.log.gz 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51747.log 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37248.log.gz 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47585.log.gz 2024-09-11T05:29:00.022 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51122.log 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51747.log.gz 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33918.log 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35435.log 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51122.log.gz 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38158.log 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33918.log.gz 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35435.log.gz 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37326.log 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50732.log 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38158.log.gz 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40295.log 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37326.log.gz 2024-09-11T05:29:00.023 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50732.log.gz 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50524.log 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40295.log.gz 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.41415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44434.log 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29695.log 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50524.log.gz 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46906.log 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44434.log.gz 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40529.log 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29695.log.gz 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46906.log.gz 2024-09-11T05:29:00.024 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29595.log 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.40529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40529.log.gz 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46802.log 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.29595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36646.log 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29595.log.gz 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51330.log 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46802.log.gz 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36646.log.gz 2024-09-11T05:29:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51330.log.gz 2024-09-11T05:29:00.750 INFO:teuthology.orchestra.run.smithi133.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-11T05:29:01.407 INFO:teuthology.orchestra.run.smithi133.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-11T05:29:01.417 INFO:teuthology.orchestra.run.smithi133.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-11T05:29:02.388 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-11T05:31:23.172 INFO:teuthology.orchestra.run.smithi133.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-11T05:32:55.517 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-11T05:33:59.228 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-11T05:33:59.229 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-11T05:33:59.229 INFO:teuthology.orchestra.run.smithi133.stderr:real 4m59.574s 2024-09-11T05:33:59.229 INFO:teuthology.orchestra.run.smithi133.stderr:user 7m0.484s 2024-09-11T05:33:59.229 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m16.895s 2024-09-11T05:33:59.230 INFO:tasks.ceph:Archiving logs... 2024-09-11T05:33:59.230 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194/remote/smithi133/log 2024-09-11T05:33:59.231 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T05:34:46.958 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T05:34:46.971 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T05:34:46.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T05:34:47.245 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T05:34:47.253 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T05:34:47.253 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T05:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T05:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2024-09-11T05:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout:*hv01.front.sepi 67.205.162.81 3 u 48 64 377 0.083 -4.464 0.416 2024-09-11T05:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout:+hv02.front.sepi 63.231.80.2 3 u 46 64 377 0.085 -3.432 0.334 2024-09-11T05:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout:+hv03.front.sepi 74.6.168.72 3 u 46 64 377 0.090 -5.522 0.406 2024-09-11T05:34:47.312 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-11T05:34:47.313 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T05:34:47.322 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T05:34:47.322 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T05:34:47.393 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T05:34:47.464 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T05:34:47.518 INFO:teuthology.task.internal:Duration was 1945.849548 seconds 2024-09-11T05:34:47.519 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T05:34:47.527 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T05:34:47.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T05:34:47.735 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T05:34:47.735 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-09-11T05:34:47.735 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T05:34:47.746 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T05:34:47.746 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T05:34:47.864 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T05:34:47.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T05:34:48.097 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T05:34:48.106 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T05:34:48.106 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T05:34:48.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T05:34:48.148 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T05:34:48.214 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-09-11T05:34:48.252 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T05:34:48.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T05:34:48.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T05:34:48.314 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T05:34:48.315 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7900194/remote/smithi133 2024-09-11T05:34:48.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T05:34:48.374 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T05:34:48.374 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T05:34:48.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T05:34:48.426 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T05:34:48.426 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T05:34:48.443 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T05:34:48.443 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T05:34:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 05:34 /home/ubuntu/cephtest 2024-09-11T05:34:48.466 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T05:34:48.475 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T05:34:48.501 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-stupid} rados supported-random-distro$/{ubuntu_latest}} duration: 1945.8495483398438 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-11T05:34:48.501 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T05:34:48.601 INFO:teuthology.run:pass